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Client satisfaction was considered to be a critical indicator of the effectiveness of 
the services provided by the Navy Regional Data Automation Centers (NARDACs). 
The purpose of this thesis was to develop the means to measure this indicator. 

Interviews of twenty-eight middle management clients served by NARDACs in 
four geographical regions within CONUS were conducted. Forty-four items that 
influence satisfaction were identified belonging to eight factor dimensions of the 
client satisfaction domain. The Likert Scale methodology was employed in the 
construction of the measurement instrument. The instrument was reviewed for 
completeness of items and logical soundness of operations by three independent 
groups of experts, thus achieving content validity. A client satisfaction index is 
formulated from data obtained with the measurement instrument. 

The instrument was piloted to a NARDAC site consisting of fifty-two client 
organizations using self-administered mail surveys. Evaluation of the instrument 
was performed on the scored results of twenty-six pilot respondents. The pilot 
results were subjected to empirical tests for construct validity and reliability. The 
results indicated that the instrument could be used to measure client satisfaction after 
further piloting on a larger (at least two hundred twenty) sample size. 
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I. INTRODUCTION 



When you can measure what you are speaking about, and express it in 
numbers, you know something about it; but when you cannot express 
it in numbers, your knowledge is of a meager and unsatisfactory kind; 
it may be the beginning of knowledge, but you have scarcely, in your 
thoughts, advanced to the stage of science. [Cited in Ref. 1 : p. 89] 

This observation by the renown British physicist, William Thompson (Lord 
Kelvin), may have been foremost in the minds of the Naval Data Automation 
Command (NAVDAC) corporate management during the development of a long-term 
business strategy for their field activities, the Navy Regional Data Automation Center 
(NARDACs). The steering committee, composed of the NARDAC Commanding 
Officers and top management in NAVDAC, was established as the NARDAC Board 
of Directors and chartered to address strategic planning matters and issues affecting 
the NARDACs as a result of a 1986 Coopers and Lybrand study of Navy Industrial 
Fund (NIF) activities. 1 

One of the issues that the NARDAC Board of Directors identified in the 
corporate long-range business plan was a need for a standardized index which would 
be used to measure the level of client satisfaction at the various NARDACs. The 
index would serve the purpose of providing an objective evaluation of each 
NARDAC's performance and its ability to deliver quality service to its clients. Also, 
as a measure of effectiveness, the index would be the cornerstone of their business 



1 The Secretary of the Navy (SECNAV) contracted Coopers and Lybrand to conduct 
a management analysis of all activities under the NIF program. The purpose of the study 
was to assess the NIF program’s effectiveness and to find any areas of deficiency. 
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strategy to increase the NARDAC's competitive posture. This would be 
accomplished by accurately evaluating their current level of client satisfaction to 
enable the establishment of a base index from which management can take 
appropriate actions to make improvements. 

Presently, a standardized measurement of effectiveness that empirically 
represents user satisfaction among those clients serviced by the NARDACs does not 
exist. Each NARDAC, operating as a cost center, is responsible for marketing its 
services to activities normally located within its geographical area of cognizance. As 
a result, the NARDACs currently assess client satisfaction using different methods, 
criteria, standards and metrics. This, in effect, has made it difficult for NAVDAC to 
obtain an accurate assessment of client satisfaction at each of their NARDACs. 

A . EVOLUTION OF THE REQUIREMENT 

1. Background 

The NARDACs were formed as a result of the Navy Automated Data 
Processing (ADP) Reorganization study and implementation plan with the basic 
objective to improve the management and operation of the Navy’s mission support 
ADP program [Ref. 2]. These field activities of NAVDAC were established to 
provide non-tactical ADP technical assistance and operational support to Navy 
activities ashore and afloat. Department of Defense (DOD) components and other 
Federal agencies. There are nine (9) NARDACs geographically dispersed within the 
continental United States (CONUS) and in Hawaii (See Figure 1.1). Although each 
NARDAC may vary in the ADP support it can provide, major functional services are 
offered at every site (See Appendix A). 
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NAVY REGIONAL DATA AUTOMATION CENTERS 




Figure 1.1 NARDAC Installations Map 
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2. History of Operations 

Prior to fiscal year 1984, the NARDACs were mission-funded which 
meant that every NARDAC received an annual allotment of appropriated funds to 
cover its costs for the support provided to its clients. Based on that operational 
philosophy, ADP products and services were delivered at no cost to the clients. 
Because this ADP support was perceived by the clients as free, there was little 
concern on their part for costs, only the timeliness and quality of the requested 
support. In that regard, the NARDACs' primary mission was focused on providing 
ADP support to their clients within the planned budgets. The need to formally 
measure client satisfaction, therefore, was not a paramount issue at the time. 

3. Present-Day Operations 

Since fiscal year 1984, the NARDACs' cost of operations has been 
financed under the Navy Industrial Fund (NIF), a revolving fund from which 
working capital funded by Congress to the Department of Defense is allocated to the 
Navy as an appropriation. The initial funds by the NIF appropriation to the 
NARDACs were used to finance the costs of providing ADP services ordered by 
their clients. In return, the products furnished and services rendered by the 
NARDACs to other Navy activities and government agencies are performed on a cost 
reimbursable basis at standardized fiscal year rates. 1 It is therefore the responsibility 
of the client to submit a Program Objective Memorandum (POM) and budget 



1 Standardized fiscal rates refer to stabilized rates for products and services that are 
fixed for an entire fiscal year. The period of a fiscal year is from 1 October to 30 
September. 
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accordingly for needed ADP services. 1 The clients pay the NARDACs through the 

use of a reimbursable order citing their own appropriated funds. In theory, the 

payment or reimbursement would then put the corpus of the revolving fund back to 

where it started. The financial operations of the NARDACs are comparable to that of 

private enterprises, i.e. working capital and clients are required. The NARDACs and 

other NIF activities, however, are run on a non-profit basis. 

Working with a NIF activity is much like contracting with a contractor 
— the only significant difference is that the NIF activity is not out to 
make a profit for stockholders. [Ref. 3: p. H22] 

Moreover, the importance of how effective a NARDAC operates cannot be 

understated. The less effective the NARDAC, the higher the charge for services 

[Ref. 3: p. HI]. 

According to CDR Charles Taylor, Commanding Officer of NARDAC 
Pearl Harbor, the change to NIF was made for two major reasons. First, in partial 
emulation of enterprises in the commercial sector, the intention was to discipline and 
motivate the NARDACs to provide the best possible service or risk going out of 
business. The second major reason was to provide a measure of discipline to the 
users. If services remained free to the users, as was the case when the NARDACs 
were under mission funding, there would be no incentive for the users to request 
only those services critical for mission accomplishment. Together, these two factors 
were intended to guarantee more efficient use of ADP resources in the Navy. 



1 The POM is a DOD Planning, Programming and Budgeting System (PPBS) 
document, prepared by the Services to the Secretary of Defense, which expresses total 
program requirements in terms of force structure, manpower, material and costs to satisfy 
assigned functions and responsibilities during the period of the Five Year Defense 
Program. [Ref 3: p. A1 1] 
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4. Impact of NIF on Operations 

It was clearly obvious to the NARDACs that the transition to NIF forced a 
significant change in their way of doing business. NIF had introduced new factors 
into the NARDACs' system of operations which had, until then, only applied to 
comparable organizations in the commercial sector. The principal change in direction 
was now focused on the clientele from whom revenues to continue operations were 
obtained. In addition to providing quality ADP support, the issue of operating cost- 
effectively and to remain reasonably affordable for the clients, became of paramount 
importance. Moreover, unlike other NIF activities who maintain a quasi-monopoly 
over the products and services they provide, the NARDACs were now forced to 
compete with other government agencies, as well as the private sector, to provide 
non-tactical ADP support. 

Because of the changes driven by NIF, client satisfaction has become 
critical for the NARDACs. Since all services are provided on a cost reimbursable 
basis and all operating expenses are paid for by revenues generated by the clients, the 
NARDACs must maintain a positive image and provide quality service in order to 
retain their share of the market. A client satisfaction index, therefore, could serve as 
a tool by which the NARDACs can gauge their market leverage and consequently 
make whatever adjustments necessary to remain competitive and financially 
operational. 

B. OBJECTIVES 

The objectives of this thesis are to identify the attributes relating to client 
satisfaction and to develop a standardized methodology for measuring and analyzing 
these attributes through the construction of a prototype measurement instrument and 
client satisfaction index for subsequent piloting and review at the NARDACs. 
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C. FOCAL RESEARCH QUESTIONS 

The focal issues of this thesis are: 

• What is an acceptable definition of a satisfied client? 

• What attributes should be used to construct a client satisfaction index for a 
NARDAC? 

• How should the data be gathered and in what form? 

• What metrics should be used to measure and analyze the data and how can 
these metrics be validated? 

• Can a microcomputer be used for data analysis and for maintaining 
information on customer satisfaction? If so, what software should be used? 

• How can the data gathering and analysis process be institutionalized to make it 
part of an ongoing effort to meet client needs? 

D. SCOPE, LIMITATIONS AND ASSUMPTIONS 

The scope of this thesis is limited to the development of a standardized prototype 
client satisfaction index specifically tailored to represent an empirical measurement of 
client satisfaction for the nine NARDACs. The NARDACs are located in CONUS at 
Naval Air Station (NAS) Alameda - for NARDAC San Francisco, NAS North Island 
- for NARDAC San Diego, New Orleans, NAS Pensacola, NAS Jacksonville, NAS 
Norfolk, Washington Navy Yard - for NARDAC Washington, and Newport; and in 
Hawaii at Naval Station Pearl Harbor. Each NARDAC has a unique set of clients 
within its geographical region. The research focuses primarily on those attributes 
relating to client satisfaction as ascertained from a cross-section of NARDAC clients. 

E. ORGANIZATION OF THE THESIS 

Chapter II presents a literature review pertinent to current thinking regarding 
client satisfaction, in private industry and in government. 



7 



Chapter III describes the methodology employed in the development of a 
measurement instrument designed as the basis for the formulation of the client 
satisfaction index. 

Chapter IV presents an evaluation of the validity and reliability of the client 
satisfaction questionnaire as a measurement instrument. 

Chapter V describes the construction of the Client Satisfaction Index (CSI) and 
several other methods of analyzing the data collected from the questionnaire. 

Chapter VI presents the conclusions of the study and recommendations for 
implementation and further research . This chapter concludes the thesis. 
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II. LITERATURE REVIEW 



A. INTRODUCTION 

The theme of this year's American Marketing Association's annual meeting was 

"The Customer Is King." The meeting focused on, and stressed the importance of 

attracting and retaining customers. Many corporations and management consultants 

have come to the same realization that the key to maximizing profits is maximizing 

customer satisfaction (e.g., [Ref. 4: pp. 2-5], [Ref 5: pp. 13-16], [Ref. 6: p. 5]). 

During the past five years of financial deregulation, banking strategists have 

increasingly concluded that one of a bank's most important assets is its existing 

customer base [Ref. 7: pp. 6, 57]. IBM believes that satisfied customers are the 

basis of their continued business success [Ref. 8]. In the words of Darryl 

Landvater, President of Oliver Wright Video Production, Inc., 

Although there are other ways to get an additional five percent of sales 
to the bottom line, many of them such as cutting expenditures for 
labor, equipment, and research and development are both painful and 
potentially damaging. It’s easier to boost sales and earnings through 
superior customer service and at the same time improve the the long- 
term position of the company. [Ref. 9: p. 86] 

Paul Allaire, President of Xerox, has recently made customer satisfaction it's 
"first priority", with return on assets (ROA) and increased market share second and 
third respectfully [Ref. 4: p. 2]. In a memo to his division directors, Allaire says, 
"we can only achieve our ROA and market share goals by satisfying our customers." 
Xerox’s new focus on customer satisfaction was the basis for the cover story, 
"Customer Satisfaction: The Big Payoff' in Xerox’s quarterly corporate publication 
Benchmark [Ref. 4]. In this article, five leading management consultants expound 
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on the importance of satisfying the customer and its payoff in increased profits. In 

the article, Robert Waterman, Jr., co-author of In Search of Excellence , cited a study 

from the Strategic Planning Institute in Cambridge, Massachusetts. 

Business sectors with a higher service content have a higher return on 
investment. The institute, which ranked similar companies in order by 
the relative quality of their products and services, found that the 
highest rated company in each area had a return on investment 100 
percent higher than the lowest-ranked company. [Ref. 4: p. 3] 

Christopher Lovelock, a former Harvard professor and principal of Christopher 
Lovelock and Associates, states that the key to improving customer satisfaction "...is 
with research to measure what characteristics of services or products are important to 
users as well as how customers perceive the goals provided by the company." [Ref. 
4: p. 3] 

Tom Peters, co-author of In Search of Excellence and A Passion for Excellence, 
believes that the qualitative aspects of business must be quantified. He believes in the 
old adage "What gets measured gets done." This philosophy is applied rigorously 
within his organization and at his executive seminars in the following manner. 



In the customer arena, we believe that regular quantitative measurement 
of customer satisfaction provides a much better lead indicator of future 
organizational health than does profitability or market-share change. 

We suggest monthly measurement. Further, we urge participants to 
make the level of customer satisfaction the primary basis for incentive 
compensation and annual performance evaluation for virtually every 
person at every level in every function throughout the organization. 

[Ref. 10: p. El] 

Clearly, customer satisfaction is an issue which is receiving considerable 
attention in the corporate community and recently it has been gaining momentum in 
the computer services industry. The purpose of this literature review is to survey 
some of the recent research that has been conducted in the area of customer 
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satisfaction with computer services and the instruments which were developed to 
quantify and measure satisfaction. The literature review is divided into three 
sections. The first section, Measures of Computer User Satisfaction, is a review of 
generic measures which could be used at any computer services organization or 
management information system/data processing (MIS/DP) department. The second 
section. Survey of Current Industry Measures, is a review of current measures of 
customer satisfaction used in the computer industry. And the last section, 
Department of Defense Measures, is a brief look at what kinds of measures of 
customer satisfaction are being used within the Department of Defense. 

B . MEASURES OF COMPUTER USER SATISFACTION 

The largest initial obstacle in conducting this literature review was finding 
information related to the measurement of customer satisfaction at a computer 
services organization. The problem was the word "customer". The computer 
industry and MIS/DP departments, in many instances, still refers to its customers as 
users instead of valuable customers [Ref. 11: p. 286]. According to W.H. Inmon, 
"Universally, data processing exists as a service organization within the company. 
The service is for the user, and the satisfaction of the user ultimately determines the 
success or failure of data processing." [Ref. 12: p. 224] 

There have been several studies conducted recently on the concept of computer 
user satisfaction or user information satisfaction (UIS). In the words of Iivari, UIS 
refers to "...a cluster of concepts that imply the assessment of information systems or 
information services in the user's subjective terms." [Ref. 13: p. 57] The 
foundations of UIS are based on the research by Cyert and March [Ref. 14]. The 
essential concept is "...that an information system which meets the needs of its user 
will reinforce satisfaction with that system. If the system does not provide the 
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needed information, the user will become dissatisfied and look elsewhere [Ref. 15: 
p.786] 

One of the first measures of UIS related to the total MIS/DP function in an 
organization which has received considerable attention was developed by S. W. 
Pearson [Ref. 16]. During his research, Pearson compiled a list of thirty-six factors 
relating to computer user satisfaction based on a literature review of twenty-two 
studies of computer-user interactions. These factors were then reviewed by three 
DP professionals for completeness and accuracy. As a result of the review, two 
additional factors were added to the list. Next, the list of thirty-eight factors was 
compared, using a critical incident analysis technique, to taped interview responses 
from thirty-two middle manager users in eight different organizations. This step 
concluded with the addition of one factor, making a total of thirty-nine distinct 
factors. These factors were then incorporated into a questionnaire which utilized the 
semantic differential technique. Each factor is measured by three separate scales 
using seven intervals from negative to positive responses. The first scale was a 
measure of one's perception by rating four bipolar adjective pairs. The second scale 
was a satisfactory - unsatisfactory pair and the third scale measured the importance of 
the factor to the user. The importance rating is used as a weighting factor for the 
overall satisfaction score. The description of the scoring method can be found in 
Reference 17. The questionnaire was empirically tested for validity and reliability 
using the same thirty-two middle managers previously interviewed. The instrument 
was found to be reliable and valid, based on the twenty-nine returned questionnaires. 
See Reference 17 for the results of the validity and reliability evaluation of the 
questionnaire. 
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A study by Ives, Olson and Baroudi was a continuation of the Pearson study 
with the emphasis to reduce the length of the overall measure and reinforce the 
validity of the instrument with more extensive testing [Ref. 15]. The results of their 
study reduced Pearson's thirty-nine original factors down to twenty-one factors with 
greater validity using a larger sample size. Description of their results can be found 
in [Ref. 15]. Ives, Olson, and Baroudi’s work was based on the following 
definition of UIS: "...the extent to which users believe the information system 
available to them meets their information requirements." 

Mathew & Co., a Data Processing Management consulting firm, has developed a 
reporting system for measuring DP-user satisfaction called "How're We Doing?" 
[Ref. 18: p. 10] The system requires users to annotate in a log, during a one month 
period, any problems with data processing based on nine "performance-related 
criteria." At the end of the month, the data from the log(s) are processed by Mathew 
& Co. which supplies the client with DP user satisfaction analyses in several 
different categories. The system is based on exception reporting (i.e., only problems 
with data processing are reported). The nine criteria are timeliness of output, quality 
of output, online availability, response time, systems development schedules, 
response to problems, and attitude and cooperativeness. Mathew & Co. provides its 
clients with reports on DP-user trends for the current month and trends over periods 
of three months, six months and twelve months. 

C. SURVEY OF CURRENT INDUSTRY MEASURES 

The banking industry recently began using a customer service index to measure 
and manage customer service [Ref. 7]. One index is based on twenty critical 
attributes of "Good Customer Service" according to a nationwide survey of bank 
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customers. Bank customers were asked to score all twenty attributes of their 
financial institution on a six- point scale, (6 = excellent). 1 

Keeping the customer satisfied with quality products and services is the 
challenge changing the face of the automotive industry also. The automotive industry 
uses a consumer satisfaction index to evaluate consumer demands (e.g., increasing 
showroom hours). [Ref. 19] 

Sitmar Cruises attributes its highest rate of repeat passengers of any major cruise 

line to its "intensive customer research" using a 1 6-page customer survey [Ref. 4: p. 

4], Benjamin French believes random surveys are the best way of determining the 

effectiveness of an organization's customer satisfaction program. 

A brief questionnaire and accompanying letter of explanation are 
simple to prepare and provide an inexpensive way of taking the pulse 
of your customers. In addition to providing you with data on your 
effectiveness, the survey also conveys your concern for solving 
customer’s problems and gives you another opportunity to restore their 
faith in your company and its products or services. [Ref. 20: p. Ill] 

However, James Carman, Professor of Business Administration at the Graduate 
School of Business, University of California at Berkeley, argues that surveys must 
be interpreted carefully because it is difficult for some customers to evaluate the 
quality of the service they received if they are lacking knowledge in the field. [Ref. 
4: p. 5] 

IBM distributes a survey to every one of their customers at least once a year in 
order to directly ascertain information relating to customer satisfaction and to improve 
their business relationship. The corporate offices send out periodic surveys to clients 
requesting their service perceptions. This is done independently from the regional 



1 The index was developed by Financial Products Group, a Chicago-based consulting 
firm serving the financial services industry, with the assistance of Market Facts, Inc. 
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offices. In fact, regional offices don't even know when this occurs. The survey is 
divided into eight major categories: quality of products, hardware maintenance and 
support, systems management, marketing support, technical support, education, 
telephone coverage, and overall rating. The survey is scored on a five-point Likert 
Scale (very satisfied to very dissatisfied) with a block for no opinion. In addition to 
the survey, there is a corporate-wide complaint system where client complaints are 
assigned to a high-level manager for investigation and follow-up. [Ref. 8] 

At Electronic Data Systems Corporation (EDS), they believe feedback from the 
customer is an important step in maintaining a high level of customer satisfaction. 
EDS North American Commercial Group uses an extensive series of questionnaires 
as part of their Quality Enhancement Program (QEP) 1 to keep abreast of their 
customers needs and to continue to improve services and products. [Ref. 21] 

D. DEPARTMENT OF DEFENSE MEASURES 

The Philadelphia Naval Shipyard, a NIF-funded activity, does not use any 
proactive initiatives to ascertain customer satisfaction. Their criteria for measuring 
performance is based on adhering to project budget and schedule constraints 2 . The 
Military Airlift Command (MAC), an Air Force industrial funded activity, does not 
actively solicit information from their customers regarding customer satisfaction 
either. Instead, suggestion/comment forms are made available to MAC customers. 3 



1 The contents of the questionnaire and the specifics of the QEP implementation 
policies and procedures are proprietary information of EDS North American Commercial 
Group and will not be disclosed in this thesis. 

2 Interview with Mr. Monteleon, Management Planning Division, Philadelphia Naval 
Shipyard, Philadelphia, Pennsylvania, 5 February 1988. 

3 Interview with CDR Jordan, USN, Navy Liaison Officer, Military Airlift Command 
Headquarters, Scott Air Force Base, 5 February 1988. 
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The Navy Public Works Centers (PWC), which are NIF-funded activities, have 
recently initiated a customer satisfaction evaluation program [Ref. 22]. The program, 
"Customer Evaluation of Support Provided by PWC", consists of a fifty-item 
questionnaire divided into six categories. The respondent is requested to rate the 
quality of service provided by the PWC using a seven-point scale. The questionnaire 
is scored in the following manner: 

a. For customer responses where the "X" is placed in blocks 6 or 7, 
score 2 points; 

b. For customer responses where the "X" is placed in blocks 3, 4, or 
5, score 1 point; 

c. For customer responses where the "X" is placed in blocks 1 or 2, 
score 0 points. 

The best possible score is 100 points. To determine the annual grade 
from all of the customers who received a questionnaire, add the total 
questionnaire scores and divide by the total number of questionnaires 
that were completed in the year. The total score is weighted by 
customer size (i.e. four surveys from large customers, two from 
medium- sized customers and one each from all others.) The result is a 
summarized customer evaluation of the support provided by the PWC 
in the past year. 

We found no published studies performed by the Department of the Navy or the 
Department of Defense in the area of customer or user satisfaction relating to 
computer services organizations or MIS/DP departments. 1 



1 Searches were made through the Defense Logistics Studies Information Exchange 
(DLSIE), Defense Technical Information Center (DTIC), and DIALOG Information 
Services, Inc. 
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III. QUESTIONNAIRE DEVELOPMENT 



The corporate heads of the NARDACs identified the critical requirement for a 
client satisfaction index, thus setting the stage for the next step of our research. We 
needed to identify a logical process to follow in order to arrive at an index that 
accurately represents the client’s satisfaction with the NARDAC's performance in 
providing products and services. To represent that state of satisfaction as 
unequivocally as possible, it was imperative that the theme of our methodology be 
based on the inputs from actual NARDAC clients and their composite view of client 
satisfaction. This was important because it ensured that the fundamental building 
blocks for the construction of a meaningful index were obtained directly from 
representative samples of the source. By using this approach, we felt that our 
methodology would be sound in terms of face validity and the resulting product 
would be realistic and more readily accepted by the NARDACs and their client 
community. 

From a study by Pearson, the measurement of client satisfaction mirrors the 
measurement of an individual's attitude or evaluative feelings toward some feature of 
the subjects of interest [Ref. 16]. There are various ways to collect data for 
measuring a client’s attitude. One basic means is by interrogation which involves 
getting people to answer questions, either in person or through telephone interviews. 
Another interrogation technique, more widely used for measuring a person's attitude, 
involves an instrument, in most cases a questionnaire, which is one of the 
fundamental tools used in survey research. The reason for the popularity of the 
survey research method is due to the numerous benefits it offers. 
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• It provides a method for empirical verification of data. 

• The data gathered by this method becomes a source of information which can 
be conveniently stored through automated methods and can be analyzed shortly 
after collection. 

• The responses can be coded into a standardized form for recording in a 
quantitative manner; therefore a standardized measurement that is consistent 
across respondents is achieved and lends itself to the development of an index. 

• Surveys can be administered fairly readily by an implementing organization (a 
NARDAC) and can be flexibly designed with minimum impact on the 
respondents' (the clients’) time. ([Ref. 23] and [Ref. 24]) 

It is because of these advantages and its applicability to the measurement of client 

satisfaction that the survey research approach was selected for this study. 

A. BACKGROUND 

During the preliminary stages of the research, background information on the 
NARDACs was gathered in order to obtain an understanding of their organizational 
structure and hierarchy, corporate culture and environment, products and services, 
and client information. This background information was obtained during site visits 
at NARDAC San Francisco, NARDAC Washington and NAVDAC. Further data 
was received from NARDAC Pearl Harbor and NARDAC San Diego. Additionally, 
the marketing representative at Honeywell was interviewed to see how client 
satisfaction was viewed from the industry perspective. We also reviewed literature 
on customer satisfaction and found current trends in the field, particularly in service- 
oriented industries. 

At the same time, we were given information on existing customer satisfaction 
methods employed by the NARDACs and Public Works Centers (PWC). Primarily, 
the methods used by these communities are informal and based on feedback from the 
client, usually obtained during personal visits or phone conversation. The PWCs 
and several NARDACs employed more formal methods, using questionnaires that 



18 



were developed in-house, as part of their marketing efforts. Some of these methods, 
extracted from a summary of findings [Ref. 25], are listed below. 



• NARDAC Jacksonville developed a client assessment form which is used 
during quarterly client visits. This assessment seems to be the most formal 
means employed to gather client satisfaction information from the client’s 
perspective. 

• NARDAC San Diego identified specific client satisfaction indices with it's 
mainframe clients. These indices are: "95% products on time" and "98% on- 
line availability during prime time and off-shift hours." Prime time availability 
is particularly significant to its major client, the Naval Aviation Depot. These 
indices are assessed and reported during regular client meetings. 

• NARDAC San Francisco recently developed an information form which will 
be used by client relations personnel when visiting clients. A portion of this 
form provides clients the opportunity to express satisfaction with or concerns 
about services. 

• NARDAC Pensacola and NARDAC San Diego use written reports of visits or 
phone contacts to identify client problems or concerns. These reports also 
provide vehicles to convey the client's satisfaction with services. 

• NARDAC Washington developed a quarterly rating report which is filled out 
by NARDAC Washington department directors. This report provides the 
department directors' expectation of a client's response to various service 
satisfaction questions. 



Because of the wide diversity in methods used among the NARDACs, it is 
understandable why the NARDAC Board of Directors recognized the need for a 
standardized method of obtaining critical client satisfaction information. 

B . FACTOR IDENTIFICATION 

Using the information gathered from interviews with NARDAC management 
personnel, relevant organizational documents and literature searches, we generated a 
list of the major products and services that the NARDAC provides to its customers. 
From this list, an initial set of factors, which was seen as measuring the NARDACs' 



19 



performance in delivering products and services, was compiled. This initial set 
consisted of the following nine (9) factors: 

• Computer capabilities 

• Ease of obtaining services 

• System design and programming expertise 

• Technical support 

• ADP consultation and assistance 

• Training 

• Cost of ADP services 

• Customer support 

• Timeliness and quality of products and services provided 

In contrast, the factors used in existing NARDAC questionnaires [Ref. 25] included: 

• Response to client requests (the quality of response and the rapidity of 
response) 

• Thoroughness of information conveyed to clients 

• NARDAC service accessibility to clients 

• Cost 

• Perception of the reliability of NARDAC personnel 

• Non-recurrence of problems and willingness to refer other organizations to 
NARDAC for computing services 

C. CONTENT VALIDATION 

In order to validate the accuracy of our initial set of factors and thereby establish 
content validity, a method was needed which was expedient, allowed for personal 
interaction with existing clients to verify the assumptions made in compiling the list 
of factors, and permitted the opportunity to identify other aspects of the NARDAC- 
client relationship which may have been overlooked or omitted during previous 
interviews with NARDAC personnel and documentation review. It was decided then 
that interviews with a representative group of active NARDAC clients would 
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accomplish this objective. A series of interviews with clients served by the 
NARDACs would be conducted within research limitations. 

1. Interview Procedures 

Because of time constraints and a ceiling on travel expenses, it was 
necessary to set criteria for determining our interview sample. This criteria included 
location, types of client organizations to interview, who within the client organization 
to interview and number of interviews to conduct. 

The first and foremost criterion was location. We wanted to focus on 
regions having a large concentration of Navy activities with differing ADP support 
requirements and where major NARDAC installations supporting these Navy 
commands offered a wide variety of computer services. The East Coast and West 
Coast regions appeared to satisfy this criterion and offered the best representative 
cross-section of respondents. The clients to be selected for interview were those 
supported by NARDAC San Francisco and NARDAC San Diego on the West Coast 
and NARDAC Washington and NARDAC Norfolk on the East Coast. 

The next criterion was types of clients to consider for the interviews. Since 
the objective in this phase of the research was to obtain information on what factors 
affect or influence client satisfaction, we wanted to interview current/active clients 
who maintained some frequency of interaction with the NARDAC providing them 
services. One way of determining the relative degree of interaction was to examine 
the amount of revenue generated from the client during the last Fiscal year. This data 
was readily available to the Client Liaison Officer at each NARDAC who compiled a 
ranked listing of clients (by revenues earned) from which candidate clients were 
selected. Although interviews with previous clients who had elected not to renew 
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services with the NARDAC were considered, we were unable to include that client 
category under this criterion because of time constraints. 

Another criterion used in narrowing the sample size of clients for the 
interview was selection of the person in the client organization with whom to conduct 
the interview. We determined that the best candidate to satisfy this criterion was the 
individual designated by the client organization as the command's official liaison or 
point of contact (POC) with the NARDAC. This individual would usually belong in 
middle management, serving as the activity's ADP Officer. Since POC information 
normally resided alongside the client organization data, this information was 
maintained and kept current by the NARDAC's Client Liaison Officer. 

The last criterion, number of interviews to conduct, was constrained by a 
number of factors: 

• Number of days available to conduct the interviews 

• Availability of the clients for the interview 

• Length of the interview 

• Travel time window required between interviews (during normal working 
hours) 

In most cases, the Client Liaison Officer of the NARDAC was in a better position to 
juggle these constraints and therefore coordinated appointments with the clients and 
arranged the interview schedule. This schedule included interviews with 28 client 
organizations, the list of which is summarized in Appendix B. Each of us 
interviewed the client organization's POC, some of whom brought in members of 
their staff to provide input during the session. 

After the appointments with the clients were scheduled, the Client Liaison 
Officer provided us a list containing the names and addresses of the clients to be 
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interviewed. Shortly thereafter, we contacted each client by phone to personally 
introduce ourselves and to explain the nature of our research. Following these phone 
calls, we mailed each client a letter containing the purpose of the interview, in 
particular, to obtain their views regarding the factors that they, as clients, would 
use to evaluate the services provided by their NARDAC. A sample letter can be 
found in Appendix C. A week before the interviews, we again called each client to 
verify receipt of the letter and to confirm the date and time of our interview. This call 
also gave the client the opportunity to ask specific questions about the interview or 
our research and allowed for any last minute rescheduling of the appointment if 
required. 

To ensure efficient use of time during the interview and to facilitate data 
collection/recording, forms were used which we had prepared beforehand to guide us 
through the interview. These forms served to assist us in collecting background 
information regarding the clients (see Appendix D) and their views on what particular 
factors contributed to their satisfaction with NARDAC services (see Appendix E). In 
addition, the forms were used to ensure consistency in the types of information asked 
of the client. In certain instances, interviews were taped but not before receiving 
permission from the client. In no instance was permission denied. 

2. Factor Validation 

During the interviews, we asked each client to comment on the nine factors 
which we had previously compiled during the process described in Section B and to 
indicate whether each factor influenced their satisfaction with the NARDAC's 
performance in providing services. The client was also asked to identify any other 
factors which were not included in the original set and did influence their state of 
satisfaction. We then requested each client to rank all the factors by order of 



23 



importance. In some cases, clients were strongly inclined to group several factors 
equally important under the same ranking. Other clients felt that some of the factors 
were not pertinent to their current requirements and therefore excluded those factors 
from their ranking. 

Following the completion of all the interviews, we compiled a tabulated list 
of factors that were ranked important to client satisfaction by each client. The 
compiled list provided us an across the board ranking of all the factors to help in 
determining which factors were considered most important by the clients. To obtain 
a composite ranking of the factors, we averaged the rankings given by the clients on 
each factor. The composite ranking of the top six factors included: 

Ranking Factor 

1 Timeliness and quality of products and services 

2 Technical support 

3 System design and programming expertise 

4 Ease of obtaining services 

5 Customer support 

6 Cost of ADP services 

We ascertained that the highest ranked factor was, in fact, two separate 
factor dimensions, Timeliness and Quality. By incorporating these dimensions as 
separate factors, a modified list of seven factors was obtained. The remaining factors 
in the original set were not discounted. They were determined to be too specific in 
scope and were therefore considered to be candidate subelements (items) relating to 
one of the seven factors. 
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3. Item Development and Validation 

Our next step was to identify specific items obtained from the interviews 
which related to the NARDAC's performance and which were measurable. The 
information was extracted from client comments documented on the form we used 
during the interviews (see Part 4 of Appendix E) and obtained in taped sessions. 
Working with our own individual set of interview data, we placed each item on a 3 
by 5 card. When this was completed, we compared our sets of items for redundancy 
and clarification, while referring to information from other surveys and relevant 
literature. Our sets were then consolidated into a set containing forty-two 
performance-related items which were then categorized under one of the seven 
factors we had identified in the previous process. In those instances when items 
could not be placed under a category because of the factor’s specificity, we redefined 
the scope of the factor and broadened its applicability to include the item. The 
factors, as modified by this process, are listed below. 

• Timeliness of Products and Services 

• Quality of Products and Services 

• Technical Proficiency 

• Hardware, Software and Communications Technology 

• Accessibility 

• Customer Support 

• Service Level Management 

The final set of items and modified factors established the client satisfaction 
domain for the NARDACs and provided the fundamental basis of what to measure in 
order to gauge the level of an individual client's satisfaction. The key elements of 
this set, which is presented in Appendix F and discussed in the following sections, 
formed the foundation for the construction of the measurement instrument. 
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D. MEASUREMENT INSTRUMENT 

Several different measuring instruments were investigated through literature 
searches and reviewing surveys currently being used in the government and private 
industry to determine which instrument would best suit our study. Buzzell, Cox and 
Brown assert, the more structured the measuring instrument is, the more accurate the 
output [Ref. 26]. Typically, in marketing measurements, it is important to have a 
measure of the degree of the response. To aid in this type of measurement, degrees 
of response are scaled to make more or less discriminations. Three of the most 
common marketing measurements are variants of the Semantic differential scale, 
Likert scales and Paired comparisons (see Figure 3.1). According to Buzzell, Cox 
and Brown, however. 

Although it is possible to attach numbers to such a verbal scale for 
purposes of analysis, it should be remembered that this is still an 
ordinal scale, strictly speaking, and estimates with regard to intervals 
or degrees of response are simply estimates. Quantify them if you 
will, but there is no direct means of determining the validity of such 
quantifications. 

There are several methods of collecting data for the instrument. The most 
common ones used in marketing research are personal interviews, telephone 
interviews and self-administered questionnaires (mail surveys). Listed in Figure 3.2 
below are some of the advantages and disadvantages of each method, as cited by 
([Ref. 27] and [Ref. 28]). The surveys that we found currently being used in the 
government and industry were all self-administered questionnaires. In the 
government, cost and time restraints seem to make self-administered mail 
questionnaires more prevalent. 

1. Questionnaire Development Standards 

Due to the limited resources available at a NARDAC to conduct surveys, 
self-administered questionnaires would be the easiest to implement. There are 
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SEMANTIC DIFFERENTIAL SCALE 



"Would you rate General Motors as being: 



Progressive 


1 


2 


3 


4 


5 


6 


Conservative 

7 


Strong 


1 


2 


3 


4 


5 


6 


Weak?" 

7 



LIKERT SCALE 

"Ajax is an excellent cleanser." 

Agree extremely strongly 

Agree fairly strongly 

Agree 

Undecided 

Disagree 

Disagree fairly strongly 

Disagree extremely strongly 



PAIRED COMPARISONS, e g., 

Do you prefer Brand A or Brand B? 
Do you prefer Brand C or Brand A? 
Do you prefer Brand B or Brand C? 



Figure 3.1 Commonly Used Types of Rating Methods and Scales. 
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1. Personal interviews : 



Advantages: 

• They tend to allow the use of a more representative sample. 

• Achieve higher return rate. 

• Produce fewer incomplete questionnaires. 

• More questions can be asked. 

• More complex measurement methods can be used. 

• Verification or responses may be made more readily. 

Disadvantages: 

• More costly. 

• Subject to interviewer bias, error and cheating. 

• Subject to response bias. 

2. Telephone interviews : 

Advantages: 

• Can be conducted quickly. 

• Relatively low cost. 

Disadvantages: 

• Sample bias. 

• Usually must be brief. 

3. Self -Administered questionnaires: 

Advantages: 

• Least costly. 

• Avoid interviewer bias. 

• Larger number of respondents can be reached. 

• More convenient to the respondent. 

• Requires a smaller staff for administering. 

Disadvantages: 

• Sample is almost certainly not likely to be "representative" unless follow 
up is done. 

• Must be carefully designed and pretested to avoid confusion on the part 
of the respondent. 

Figure 3.2 Data Collection Methods (Advantages and Disadvantages) 
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several methods of conducting a self-administered questionnaire. The questionnaire 
could be administered to a group of respondents gathered at the same place at the 
same time (e.g M at a NARDAC Client Council/Advisory Board meeting). They 
could also be hand-delivered at the respondent’s office to be completed and picked up 
at a later time. Mailing is another option, or a combination of mailing and one of the 
above methods can also be used. [Ref. 23: p. 159] The questionnaire can be hand- 
delivered, where it could be explained and any questions answered, and then mailed 
when completed. The questionnaire designed for the NARDACs was based on the 
premise that the survey would be mailed with limited intervention required by the 
NARDAC. The following criteria was used in developing the NARDAC Client 
Satisfaction questionnaire: 

• Instructions for completing the questionnaire must be clear and concise. 

• Scales must be easily understandable and unambiguous. 

• Questions will be closed-ended and as easy to answer as possible. 

• Definitions will be provided as required. 

• Comment section will be provided. 

• The number of questions should be held to an absolute minimum. 

• Questionnaire must have an overall professional appearance (i.e., neat and 
legible). 

Since the majority of the questionnaire respondents are considered middle 
to upper level management within their organization (GS-12 to GM-15), we felt a 
self-administered questionnaire would provide us with the needed information with 
minimum intrusion on their schedules. In this regard, we felt it was important that 
the instructions were self-explanatory and the questions easily answered. According 
to Arlene Fink, "A self-administered questionnaire that is hard to read can confuse or 
irritate respondents. The result is a loss of data." [Ref. 28: p. 44] We tried to 
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formulate the questions in the respondents' own vernacular. Definitions of certain 
terminology and phrases were provided to help alleviate any ambiguities. We also 
felt it was important to provide the respondent with an opportunity to comment on 
any of the items in the questionnaire or provide additional information since the 
questions were closed-ended. Additionally, the questionnaire had to make a good 
visual impression. Since the questionnaires are distributed to valued NARDAC 
clients, we felt that if the appearance made a bad first impression, then response rate 
could possibly suffer. 

2. Questionnaire Format 

The questionnaire measures the respondents level of satisfaction with respect to a set 
of forty-two performance related items. These items are grouped together into seven 
categories or factors as described in section B (Factor Identification) above. 
Although we were concerned with the length of the questionnaire, we allowed 
adequate space between questions for comments and to prevent the questionnaire 
from looking cluttered. The questionnaire is eight pages long or four pages copied 
back-to-back. It is widely acknowledged in the social research community that an 
improperly laid out questionnaire can not only confuse the respondents, but also 
make the scoring of the questionnaire more error-prone and time consuming. One of 
the most common questionnaire formats is one where the respondent is asked to 
check one response from a series [Ref. 27: p. 205]. Each item is scored using a five- 
point Likert scale. A not applicable box is also provided. According to Babbie, 
boxes, adequately spaced apart, are the best for the respondent to answer. The scale 
is measured from one to five, where five is very satisfied, and one is very 
dissatisfied. The respondents are asked to place an X in the box which most 
appropriately describes their level of satisfaction with that item. The scales are laid 
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out vertically to the side of each item to facilitate scoring of the questionnaire. Key 
words and/or phrases were underlined in each item to help focus the respondent’s 
attention on the major point the item is measuring (see Appendix F). 

3. Questionnaire Critique 

After the First draft of the questionnaire was completed, it was sent out for 
review to all nine NARDAC Client Liaison Officers/Marketing representatives, and to 
various Naval Postgraduate School faculty (see Appendix C). Generally, the 
comments from the NARDACs were favorable. Most felt we had developed a 
comprehensive questionnaire and had addressed the major issues presently 
concerning the NARDAC clients. The majority of the comments focused on the 
format and length of the questionnaire. Although some of the NARDACs indicated 
there were some questions that might be redundant, none of them indicated which 
ones were the culprits. One NARDAC felt the sentence structure of the items was 
too choppy and short, and another suggested we include an overall rating question at 
the end of the questionnaire. Surprisingly, the majority of the comments regarding 
item construction came from the Naval Postgraduate School faculty. It was pointed 
out that many of our items were asking the respondent to rate more than one aspect in 
a single item. For example, in item twelve, the respondent is asked to rate the format 
and the quality of the Chargeback Report. The respondent may feel that the format is 
satisfactory, but the quality is not. Other comments dealt with the lack of definitions 
regarding the meaning of certain terms, and the order in which the questions were 
presented. The above comments were incorporated into the second draft of the 
questionnaire. Comments referring to the implementation of the questionnaire were 
disregarded because they fell outside the scope of our study (e.g., adding an internal 
routing stamp). 
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4. Questionnaire Second Draft ( Short Form) 

The second draft of the questionnaire looked considerably different from 
the first version (see Appendix G). It is divided into three major sections. The first 
section still measures the respondents level of satisfaction with respect to a set of 
performance related items, but the three comment lines have been dropped, two 
additional questions were added, and the ordering and grouping of the questions 
were changed. These revisions made the questionnaire appear to be shorter - four 
pages long or two pages copied back-to-back ( Short Form ). The items are still 
grouped together into seven categories or factors, but the names have changed. 
Although the length of the questionnaire was a concern, we did allow adequate 
spacing between questions to prevent the questionnaire from looking cluttered. The 
response boxes formed a matrix down the right side of the page to facilitate the 
scoring of the questionnaires. The scale was repeated at the top of each page for the 
convenience of the respondent. 

A new section was also added to the Short Form. In Robert Alloway's 
paper, Defining Success for Data Processing, he discovered that in the six companies 
that he studied, all were violating the fundamental rule of management, "...identify 
which activities are most important and allocate resources to ensure good 
performance on those activities." [Ref. 29: p. 1] Thus, the second section of the 
questionnaire measures the relative importance of each of the seven major categories 
described in the first section of the questionnaire. Again, each category is scored 
using a five-point interval scale. Boxes are provided for the respondents answers. 
The scale is measured from one to five, where five is critical, and one is irrelevant. 
The use of the second section will be described in detail in Chapter V. The data from 
this section will not be used in the computation of the index; it will only be used as a 
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management tool. The last section of the questionnaire permits the respondent to 
make comments about any item(s) or general comments about the service provided 
by the NARDAC. The trailing edge of the questionnaire provides the NARDAC with 
pertinent administrative information regarding the respondent. 

5. Final Review of Questionnaire 

A revised long form 1 and the short form were sent out to all NARDAC 
Commanding Officers (COs) and NAVDAC for one last review before the pilot 
survey was conducted. During a NARDAC Board of Directors meeting, the COs 
decided that the scale identifiers at each extreme did not demonstrate enough 
differentiation from the other identifiers. Therefore, the words extremely satisfied 
and extremely dissatisfied replaced the identifiers very satisfied and very 
dissatisfied. They also changed the middle identifier from Neither Satisfied Nor 
Dissatisfied to Neutral. They felt this wording would be be easier to interpret by the 
respondents and NARDAC management. Other changes that were made included the 
definition of factors and items, the rearranging and regrouping of items, and the 
revision of some of the factor names. One additional factor was also added. The 
short form of the questionnaire was unanimously approved. They felt that the long 
form would be too overwhelming for most of their clients. However, they still 
would like to use the long form for specific clients from whom more data is required 
to help identify and resolve discrepancies. 



1 The revised long form is the original questionnaire with only content changes 
made. The format remained the same (see Appendix H). 
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IV. QUESTIONNAIRE EVALUATION 



This chapter describes the methods we employed to evaluate the measurement 
instrument, the client satisfaction questionnaire. As discussed in the preceding 
chapters, this instrument was developed to capture the domain of client satisfaction 
for the NARDACs and the distinguishable elements (items) that influence this 
domain. The administration of the questionnaire to a group of subject clients at a 
pilot site and the evaluation of the questionnaire based on the scored results are 
presented. The results were analyzed using several statistical software packages. 
STATGRAPHICS, a personal computer (PC) based statistical graphics system, and 
the Statistical Package for Social Scientists (SPSS-X), a mainframe computer 
program, were used in performing descriptive statistics, construct validation tests, 
and reliability tests. In addition, various microcomputer programs were augmented 
to provide other tools for the evaluation of the measurement instrument. 

A. PILOT DESCRIPTION 

To collect the data for evaluation of the questionnaire, we used a self- 
administered mail survey. One NARDAC site was chosen to pilot the measurement 
instrument, consisting of fifty-two client organizations which formed the sample 
population. Although we realize that a nationwide survey would provide us with a 
greater sample size to item ratio and more statistically sound test results, time and 
schedule constraints and the lack of physical resources compelled us to preclude 
exercising this alternative. Nevertheless, we felt the pilot sample size was sufficient 
for an initial evaluation of the instrument. Considering geographical proximity, the 
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clients serviced by NARDAC San Francisco were therefore selected as the subject 
group for evaluation of the questionnaire (see Appendix I). 

1. Pilot Administration 

After receiving approval from the Commanding Officer of NARDAC San 
Francisco to conduct the pilot, we obtained a list of active clients from his Client 
Liaison staff. Information from this list included the name of the client organization, 
name of the organization's POC, mailing address and telephone number. 
Additionally, the staff supplied us with supplemental profile information on each 
client. The profile data consisted of current fiscal year revenues earned to date from 
the clients and types of services provided to the clients by the NARDAC. 

Once the NARDAC Board of Directors gave us the approval to proceed 
with the pilot testing, we mailed each client a pilot survey package that consisted of a 
cover letter, the self-administered questionnaire (see Appendix J) and a pre- 
addressed return envelope. In the cover letter, we requested each client's cooperation 
in completing the questionnaire to assist us in evaluating the measurement 
instrument. The client was also asked to critique the questionnaire in terms of 
content, presentation and format. A sample of the cover letter is presented in 
Appendix C. 

The survey packages were posted to the clients ten days before the 
requested return deadline which allowed the respondents at least one full work week 
to complete and return the questionnaire. The return envelope was provided to 
accelerate the return time. 

2. Pilot Response 

Of the fifty-two survey packages mailed, we received twenty-six completed 
questionnaires which seemed to suggest a fifty percent response rate. Current survey 
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research guidelines on percentage return rates indicate that fifty percent is an 
acceptable response rate. Babbie suggests that a response rate of seventy percent or 
more is very good, a response rate of at least sixty percent is good and a response 
rate of at least fifty percent is adequate for analysis and reporting. He does caution, 
however, that this yardstick is only a rough guide which has no statistical basis. The 
accepted practice in computing response rates is to omit all those questionnaires that 
could not be delivered due to bad addresses and the like. This number is subtracted 
from the initial sample size to obtain the net sample size. Then, the number of 
completed questionnaires is divided by the net sample size to produce the response 
rate. [Ref. 23: p. 165] Although the number of questionnaires we received was far 
below the number we expected, we felt that this could have been attributed to the 
timing of the pilot survey which occurred during the Christmas season (early 
December). To substantiate reasons for adjusting the net sample size, it was 
necessary for us to obtain concrete evidence for the number of omitted non- 
responses. To accomplish this, we made numerous attempts to contact by phone 
those clients who had not responded. In many cases, we were unsuccessful in our 
efforts due to busy signals or no answer. In those few instances when we were able 
to contact the client organization, messages were left to the organization's POC, yet 
no return calls were ever received. Some of the client organization POCs who were 
successfully contacted indicated that they never received the survey package. In one 
specific case, the survey package had been addressed to the client organization 
headquarters rather than to the field activity where the organization's POC was 
located. Among the non-respondents contacted, we accounted for three clients who 
did not receive the survey package. Two survey packages were returned for reasons 
of insufficient address or unknown addressee. And lastly, one completed 
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questionnaire arrived four weeks after our analysis of the initial set of twenty-six 
responses. Our net sample size after adjustment, therefore, was forty-six which 
yielded a response rate of fifty-seven percent. 

Due to time and schedule constraints, subsequent mailings were not 
conducted. Moreover, we determined that the response rate of fifty-seven percent 
was acceptable for purposes of our evaluation. 

3. Processing of Pilot Data 

The twenty-six returned questionnaires were processed in the following 
manner. As survey packages were returned and after each envelope was opened, the 
questionnaire was reviewed and checked against the master mailing list. Each 
completed questionnaire was then assigned a unique client code. The convention 
used in assigning the client code was straightforward - a number that represented the 
sequence in which the questionnaire was received. The scored results of each 
questionnaire were then entered into a microcomputer based spreadsheet (EXCEL) 
and saved on diskettes for subsequent processing by various statistical software 
programs. The specific software programs and the results obtained from the 
processing of the scores are discussed in the next following sections. 

B. DESCRIPTIVE STATISTICS 

Basic descriptive statistics were used to evaluate the data collected from the pilot 
survey. 1 Frequency distributions were performed on each item in order to uncover 
any irregularities in the responses. What was found, was an unusually large number 
of not applicable (N/A) responses had been given for many of the items. Figure 4.1 



1 The descriptive statistics developed from the pilot survey are proprietary 
information of NARDAC San Francisco, and will not be disclosed in this thesis. 
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Figure 4.1 N/A Response Rate Per Item 



shows a histogram of all the items and their associated percent of not applicable 
responses. A three-step process was used to determine the cause of the high N/A 
response rate. 

The first step was to isolate those items that had an unusually large number of 
not applicable responses; the mean was chosen as the break-point. Any item which 
had a not applicable response rate above 3 1.4 percent was coded with an asterisk on 
the histogram as shown in Figure 4.1 (i.e., items 5, 6, 9, 17, 18, 19, 20, 21, 22, 
23, 25, 26, 27, 29, 38, and 42). Table 4.1 shows the break-down of the isolated 
items by factor. 

It was interesting to note that all of the items for the factor Quality of Products 
and Services, and three out of the five items for the factor Timeliness of Service, fell 
above the mean (3 1 .4%) N/A response rate. Problems discovered in item 
construction for these two factors will be discussed in Section D of this chapter. 

The next step was to see if there were any relationships between the sixteen items 
listed in Table 4.1 and the respondents which contributed significantly to the N/A 
response rate for those items. Figure 4.2 shows the number of N/A responses for 
the sixteen isolated items by respondent. Again, the mean was chosen as the break- 
point to separate those respondents that had a significantly large number of N/A 
responses. The asterisks in Figure 4.2 indicate which respondents fell above the 
mean (9) rate (i.e., respondents 1, 2, 3, 4, 5, 7, 8, 11, 12, 13, 21, 22, 23, 24, and 
25). 

During the last step, we looked at the profile of the respondents isolated in the 
previous step to determine what may have caused these respondents to answer N/A to 
so many items. Table 4.2 lists all the pilot respondents (by code), NARDAC San 
Francisco’s fiscal-year-to-date (FYTD) revenue for that respondent, and the type of 
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* Respondents Scoring N/As Over Mean 



TABLE 4.2 Pilot Client Profile Data 



CLIENT CODE 


FYTD REVENUE ($000) 


SERVICE PROVIDED 


1 


23 


Risk Assessment 


2 


5 


Training 


3 


2 


Study 


4 


1 


Programming 


5 


0 


- 


6 


46 


Training & Procurement 


7 


2 


Training 


8 


17 


Training 


9 


10 


IRC Support/Sperry 


10 


5 


Labor 


11 


14 


Study & Procurement 


12 


1 


Programming 


13 


117 


ADP Security Risk 






Assesment 


14 


26 


Programming 


15 


24 


IRC Support, Procurement, Tech 






Support 


16 


172 


Programming, Telecom, Key- 






Entry, Burroughs, IV Phase 


17 


6 


FYPR-Sperry 


18 


34 


Sperry, Comlines, Terminal 


19 


1 


Sperry 


20 


5 


Labor, Misc, Telecom 


21 


21 


Procurement, Training 


22 


8 


IRC Support, Procurement 


23 


293 


Programming, Procurement 


24 


6 


Training 


25 


0 


- 


26 


17 


IRC Support, Procurement 


Mean 


32.9 


For all respondents 


Mean 


7.7 


(Except for respondents 13 






& 23) 


Sum 


856 


For all respondents 
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service provided. The respondents in bold type are the ones isolated in step two. 
The following observations were made: 



• The mean FYTD Revenue for all the isolated respondents (except respondents 
13 and 23) 1 was $7,700. As compared to the total mean for all respondents, 
these are relatively small clients. 

• Forty percent of the highlighted respondents only received training as a 
service. The questionnaire does not accommodate training services as well as 
other services provided by the NARDAC. This problematic area will be 
addressed in Section D of this chapter. It should also be noted that many of the 
NARDACs distribute separate surveys for clients receiving training. 

• The contract for respondent 3 had not begun at the time the pilot was 
conducted. 

• The contracts for respondents 5 and 25 have been cancelled. 



TABLE 4.1 Break-down of Isolated Factors. 



FACTOR 


ITEM(S) 


Accessibility 


5 


System Resources 


6,9 


Cost Management 


- 


Quality of Products and Services 


17, 18, 19, 20,21,22, 23 


Timeliness of Services 


25, 26, 27 


Responsiveness 


29 


Staffing 


38 


General Business Practices 


43 



1 Respondent 13 had a one time contract for ADP Security Risk Assessment and had 
little interaction with NARDAC San Francisco. Respondent 23 had a large hardware 
procurement which accounted for 75% of his total FYTD revenue. It was felt that these 
two respondents' relatively high FYTD revenue was not the norm and were disregarded in 
the computation of the mean for the isolated respondents. 
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It would be premature to make any widespread conclusions about the quality of 
the questionnaire from this phase of the evaluation since the number of respondents 
(26) from the pilot survey was small. Further testing of the instrument with a larger 
sample size will be required in order properly evaluate the questionnaire; however, it 
is clear that there are problems with some of the items in the factors Quality of 
Products and Services and Timeliness of Services. These issues will be addressed in 
Section D. 

Generally, the comments about the questionnaire were quite favorable. Only 
seven out of the twenty-six respondents actually commented on the construction of 
the questionnaire. Of those who commented, over seventy percent thought the 
questionnaire was well prepared. A typical comment was, "Your questionnaire was 
quite thorough, easy to read, and concise.. .the instructions were fine...". Only one 
respondent (#12) felt the questionnaire was too long, but he is a new client and 
answered thirty-one of the forty-four items on the questionnaire with a not applicable 
response. 

C. VALIDATION AND RELIABILITY PROCEDURES 

The next step in the evaluation of the instrument was to test the questionnaire for 
its validity and reliability. In this study, we examined content validity or face validity 
and construct validity. Cronbach's alpha was used to examine reliability. 

1. Content Validation 

Content validation is determined from the content and operations of the 
measurement instrument [Ref. 15: p. 156]. Others, such as Babbie, refer to content 
validity as face validity or logical validity. For instance, if we were indeed interested 
in measuring client satisfaction of those clients supported by the NARDACs, then 
from a logical point of view each of the items considered should appear on its face to 
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indicate the respondent's satisfaction with the services provided by the NARDACs. 
[Ref. 23: p. 256] The content of the measurement instrument, therefore, has to be 
designed to include all important items or attributes affecting a client's satisfaction 
with the NARDAC support. As described in Chapter III, this has been the 
underlying theme of our methodology to identify those items and to test the list of 
items for completeness. 

The items were examined, prior to the pilot, by independent groups of 
experts as the initial phase in validating the completeness of the items list. This 
follows a similar approach used by Pearson in which independent assessments and 
the collective experience of expert groups provided different perspectives to 
counterbalance any significant omissions that could have occurred. [Ref. 16: p.89] 
The first group of experts was selected on the basis of their experience in ADP and 
interaction with the clients. This group was composed of all the NARDAC Client 
Liaison Officers who reviewed the initial draft of the questionnaire. The same 
questionnaire was reviewed by another group of experts in the academic environment 
with research and consulting experience in the fields of data processing, survey 
research and statistics, and organizational management. This second group was 
comprised of faculty in the Information Systems, Operations Research and 
Administrative Sciences curriculums at the Naval Postgraduate School. Based on the 
comments and recommendations from the first two groups of experts, the item list 
was modified and the questionnaire revised. The revised questionnaire was then 
reviewed by a third group of experts with extensive experience in the data processing 
field and management in the Navy and with a history of formal interaction with client 
organizations. This last independent body of reviewers was made up of the 
Commanding Officers of the NARDACs and the NAVDAC staff, in essence, the 
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NARDAC Board of Directors. Details of these reviews are described in Chapter III, 
Section D, Parts 3 and 5. The modifications recommended by the three review 
groups and the independent assessment of the completeness of the item list indicated 
that the important items influencing a client's satisfaction with NARDAC services 
had been identified and validated. 

The other aspect of content validity deals with the logical soundness of the 
operations for measuring the content of the items. To establish the operations for the 
measurement of this content, the Likert Scale methodology was used. The selection 
of this methodology for appropriateness and the development of the measurement 
instrument is described in Chapter III. On the basis of this approach, we determined 
that this measurement process was further evidence of content validity, although 
subjective in nature. 

Content validity was likewise performed by the clients themselves during 
the pilot phase of the measurement instrument. The clients participating in the pilot 
were asked to critique the questionnaire by providing an assessment of the following: 

• Garity of item phraseology 

• Appearance of item redundancy 

• Omission of critical items 

• Readability of the survey and ease of completion 

• Garity of the instructions 

Their assessment of the above items addressed the questionnaire's content structure 
and completeness of items; therefore, further evidence of content validity had been 
achieved. 
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2. Construct Validation 

In the previous section, content validity of the measurement instrument and 
the process in performing this test for validity was described. Construct validity 
focuses on the nature of the items being measured and the extent to which these 
particular items relate to one another. Thus, it attempts to measure the correlations 
among many independent items to determine whether these items are strongly enough 
related to describe a particular relational concept. In this case, construct validation 
was used to determine how strongly each of the forty-four items related to one 
another and to each of the eight factors. Factor analysis was the technique employed 
in an effort to perform this type of validation. 

a. Factor Analysis Description 

Factor analysis is a multivariate statistical tool that analyzes 
interrelationships among many items (e.g., questionnaire responses) and then 
explaining these items in terms of their common underlying dimensions (factors) 
[Ref. 31: p. 427]. It is, in effect, a simultaneously processed item interdependence 
technique that tells which item responses measure the same factor and to what extent 
they measure these factors. 

In examining the pilot responses for construct validity, we performed 
factor analysis on the forty-four item scores for each of the twenty-six respondents. 
This was accomplished using the factor analysis software module in 
STATGRAPHICS on a Zenith 248 microcomputer and the SPSS-X factor analysis 
program on an IBM 3033/4381 computer. The purpose of these trials was to 
determine if there was indeed some logical pattern among the forty-four items 
intercorrelations and to see what dimensional factors would be generated based on 
the intercorrelations of these empirical input. STATGRAPHICS allows the use of 
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the original item responses and prompts for the number of factors to extract in the 
analysis. SPSS-X performs similarly and, as an addition, provides the feature of 
generating the number of factors automatically. 

b. Factor Analysis Procedures 

The forty-four item scores for each of the twenty-six respondents were 
entered into a file for processing by the statistical software programs. In the case of 
STATGRAPHICS, these entries were simultaneously stored by the software 
program to the work diskette. Parameter specifications used for the factor analysis 
runs were Pairwise (to handle N/A items) and Varimax rotation. The output in 
Pearson Coefficient format (available on SPSS-X only) contained more meaningful 
information for evaluation. The interested reader is referred to the 
STATGRAPHICS User Guide [Ref. 32] and SPSS User Manual [Ref. 34] for more 
details on the various factor analysis parameter options. 

c. Interpretation of Factor Analysis Output 

The output of a factor analysis program is presented in matrix form and 
consists of several key components. The columns represent the factors (artificial 
dimensions) generated from the observed relations among items. The values under 
each factor column represent the correlations between each item and each factor and 
is referred to as the factor loadings. In examining the output, one can determine the 
meaning of a given factor based on those items that load highly on it. Babbie points 
out that the generation of factors, however, has no reference to the meaning of the 
item, only their empirical associations. Furthermore, he offers two important criteria 
to consider while evaluating this data. 
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• A factor must explain a relatively large proportion of the variance found in the 
items, and 

• Every factor should be more or less independent of every other factor. [Ref. 
23: p. 328] 

Although the interpretation of the complex interrelationships found in 
the factor analysis matrix output is no simple matter, the following provides a step- 
by-step method for evaluating the data. 1 



1 . Examine the factor matrix. Each column denotes a separate factor. The values 
beneath the columns are the factor loadings for each item on each factor. The 
numbers on the left margin of the matrix represent each of the 44 items in the 
questionnaire. 

2. Begin the analysis by starting at the first item on the first factor and move 
horizontally from left to right, looking for the highest loading for that item on 
any factor. For sample sizes less than 100, the lowest factor loading to be 
considered significant would be ± .50. If the highest loading is significant, 
underline it. 

3. Proceed to the second item and, again moving from left to right horizontally, 
look for the highest loading for that item on any factor and underline it. 
Continue the procedure for each item until all the items have been underlined 
once for their highest significant loading on a factor. Some items may have 
several loadings. 

4. Identify items that have not been underlined (those that do not load on a factor). 
If the item is considered important, leave the item as is. If the item(s) are 
considered of minor importance to client satisfaction, the item(s) may be 
eliminated and derive another factor analysis solution with the non-loading 
items eliminated. 

5. When all significant items are loading on a factor indicating that a factor 
solution has been obtained, assign a name or meaning to the pattern of factor 
loadings. Items with higher loadings are considered more important. The final 
result will be a label or a name that represents each of the derived factors. [Ref. 
34: pp. 250-251] 



1 It is recommended that the expertise of a social researcher be used in interpreting the 
factor analysis output. 
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d. Pilot Results 

The factor analysis execution runs of the pilot data using both 
STATGRAPHICS and SPSS-X produced unusable output and therefore 
inconclusive results. One explanation for these results could be the small sample size 
to item ratio for this pilot (.59 : 1). Hair et. al. emphasizes that, as a general rule, 
there should be five times as many responses as there are items (5:1) to be analyzed 
and that this ratio is considered to be somewhat conservative. He further adds that 
when dealing with smaller sample sizes and a lower sample size to item ratio, any 
findings should be interpreted with caution. [Ref. 34: pp. 250-251] Ideally then, a 
preferred sample size to ensure a more meaningful and empirically sound evaluation 
of the questionnaire should be at least 220 (i.e., 5 times 44 items). 

3. Reliability Testing 

According to Kerlinger, "reliability can be defined as the relative absence of 
errors of measurement in a measuring instrument". In other words it is the accuracy 
or precision of a measuring instrument. [Ref. 30: p. 405] There are basically two 
types of reliability tests that can be performed: test-retest method, and the internal 
consistency method (Cronbach's alpha). According to Carmines and Zeller, test- 
retest method is one of the easiest ways to estimate the reliability of empirical 
measurements [Ref. 31: p. 37]. In this method, the same survey is given to the same 
set of respondents after a period of time, and the correlation between the scores is 
obtained. If exactly the same results are obtained on the two administrations of the 
survey, the test-retest reliability coefficient will be 1.00 (i.e., perfect reliability). The 
problem with this method is that the respondent’s perception of client satisfaction 
will presumedly change over time based on the quality of services provided by the 
NARDAC. Thus, a low test-retest reliability correlation may not indicate that the 
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reliability of the questionnaire is low. The test-retest method can also be expensive 
and impractical to administer. 

A much better method of determining reliability is the measurement of the 
amount of error in the instrument. The amount of error in a measure can be 
determined using Cronbach’s alpha test applied to interitem correlations, which can 
be expressed as follows: 



a 



Np’ 

where 

[1 + p'(N - 1)] 



N = the number of items 

p' = the mean interitem correlations. 



The value a will vary between .00 and 1.00, when the mean interitem 
correlations 1 are between zero and one. The value a at 1.00 is perfect reliability, but 
a reliability score of .80 is considered acceptable for basic research [Ref. 15: p.788]. 
As with the factor analysis procedure, the reliability test was performed, but because 
of the small sample size, the results were inconclusive. It is recommended, however 
that a reliability test be performed on the instrument prior to implementing the Client 
Satisfaction survey. The ratio of sample size to number of items should be at least 
(5:1) or greater. Reliability testing is not available in the STATGRAPHICS program 
however; it can be easily tested using procedure RELIABILITY on SPSS-X. The 
Alpha model (Cronbach's alpha) is the default model in procedure RELIABILITY. 



1 To find the mean interitem correlation, the correlation coefficients are summed and 
divided by the total number of coefficients. 
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D. REVISED QUESTIONNAIRE 

The frequency distributions calculated in section B above highlighted two factors 
that displayed an usually high rate of not applicable responses: Quality of Products 
and Services and Timeliness of Services. Each item within these two factors were 
carefully examined and compared to the profile of the respondents which answered 
not applicable to those items. From our analysis, we concluded that clients, such as 
those receiving training or Information Resource Center (IRC) 1 assistance, had 
difficulty answering the questions in those two factors. 

As a result, questions 18 and 28 were identified as being too specific and were 
revised to broaden their applicability. Figure 4.3 illustrates the changes made. Of 
course, further testing of the instrument with a larger sample size will have to be 
performed to further refine the instrument, but we feel these preliminary changes will 
reduce the number of not applicable responses for the factors, Quality of Products 
and Services and Timeliness of Services. See Appendix K for the revised 
questionnaire. 



1 The IRC at NARDAC San Francisco was created to help Navy commands and 
other government agencies to deal with the microcomputer revolution. Basically, it helps 
the client take full advantage of the productivity enhancement capabilities of a 
microcomputer. 
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QUALITY OF PRODUCT AND SERVICES 

(NARDAC's ability to deliver reliable services and excellent products) 



CURRENT: 18. The quality of NARDAC's data processing services. 

REVISED: 1 8. The quality of NARDAC's services . 

(Services include training, Information Resource Center (IRC) 
assistance, data processing, Client Liaison services, studies, etc.) 



TIMELINESS OF SERVICES 

(NARDAC's ability to be punctual and ”, schedule conscientious" with its products 
and services) 



CURRENT: 28. Timeliness of deliverables from NARDAC. 

REVISED: 28. Timeliness of services from NARDAC. 

(Services include training. Information Resource Center (IRC) 
assistance, data processing, Client Liaison services, studies, etc.) 



Figure 4.3 Changes to items 18 & 28. 
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V. DATA ANALYSIS TECHNIQUES 



A. INDEX CONSTRUCTION 

An index is a composite measure very frequently used in social research [Ref. 
27: p. 361]. Babbie lists the following advantages of an index: 

• Allows a researcher to develop a composite measure of variables. 

• A single data item might not have enough categories to provide the desired 
range of variation, but an index formed from several items would. 

• Indexes are efficient data reduction devices : several indicators may be 
summarized in a single numeric score. 

An index is constructed by accumulating scores assigned to individual attributes. 
The data from the client satisfaction questionnaire was formulated into an index, 
which would empirically describe the level of client satisfaction. The Likert scale 
was quantified by assigning values 5, 4, 3, 2, and 1 to extremely satisfied, satisfied, 
neutral, dissatisfied, and extremely dissatisfied respectfully; the not applicable scores 
were disregarded in constructing the index. According to Babbie, "the Likert format 
lends itself to a rather straightforward method of index construction. Since identical 
response categories are used for several items intended to measure a given variable, 
each such item can be scored in a uniform manner." [Ref. 27: p. 375] Using the 
values above, the scores on the questionnaire were averaged for each factor to arrive 
at a factor index: 




k=l 



where 



Fj = individual factor index 

nij = total number of items for factor j 
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L = number of applicable responses 
Ijj = numeric score for item k of factor j, 

= 1,2, 3, 4, 5. 

To determine the overall Client Satisfaction Index (CSI), all of the factor indices (Fj) 
are averaged: 



CSI = tt F: where 

i=l 

N = number of meaningful factors 

(i.e., factors where at least one item was applicable) 
i = the number of factors. 

The result will be a CSI value between one and five. Each factor was given an equal 
weight because there were no substantiating reasons for differential weighting of the 
factors. An interpretation of the index is shown in Table 5.1. 



TABLE 5.1. 



CSI RANGE VALUES 


INTERPRETATION 


4.21 to 5.00 


Extremely Satisfied 


3.41 to 4.20 


Satisfied 


2.61 to 3.40 


Neutral 


1.81 to 2.20 


Dissatisfied 


1.00 to 1.80 


Extremely Dissatisfied 



B . IMPORTANCE-PERFORMANCE GRID 

The Importance-Performance Grid was developed by Robert Alloway, from the 
Sloan School of Management, Massachusetts Institute of Technology, to help 
identify which client satisfaction factors are most important and to allocate resources 
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to ensure good performance on those factors (i.e., "prioritizes management's 
attention"). [Ref. 29] 



The last section of the questionnaire is used to collect data which can be used to 

develop an importance-performance grid. The respondent is asked to rate the relative 

importance of each of the eight performance related factors on a scale of one to five. 

These importance scores are then plotted against the average performance scores for 

each factor (Fj). After all the factors have been plotted, the grid is arbitrarily divided 

into four quadrants. The placement of the axes is a managerial decision based on 

what senior management feels are achievable standards of success (e.g., see Figure 

5.1). Alloway defines the quadrants as follows: 

The upper-right comer is the relative success quadrant. This implies 
no change is necessary in importance-performance for these criteria. 

The lower- left quadrant is also OK in the sense that lower performance 
on these comparatively unimportant criteria is acceptable. My 
recommendation for these two quadrants, leave them be, might appear 
much too benign until one contemplates the level of managerial 
attention and effort required to improve the lower-right quadrant. 

The lower-right quadrant is the real killer. These criteria have high 
importance but low performance. These are the criteria which ruin a 
DP department’s reputation, drive users up the wall, seriously impair 
DP's ability to deliver, and prevent user managers from receiving their 
relevant information. 

The upper-left quadrant should receive declining management 
attention. Any increased efforts to improve performance here are a 
relative waste of resources. Clearly DP management should not steal 
resources from the success quadrant for use in the killer quadrant, 
rather, DP should reallocate from waste and OK to killer. [Ref. 29: 
pp. 15-17] 

Each client will have a unique profile based on his Importance-Performance Grid 
that will focus management's attention on the factors that the client feels requires 
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greater attention by the NARDAC. It will also assist the NARDAC in managing 



scarce resources. 



a FACTORS 

1 - Accessibility 

2 - System Resources 

3 - Cost Management 

4 - Quality of Product 
and Services 

5 - Timeliness of Services 

6 - Responsiveness 

7 - Staffing 

8 - General Business 

• iii« Practices 

0 1 2 3 4 5 6 

IMPORTANCE 



tu 

0 
z 
< 

1 

2 

£ 

a 



4 - 



3 - 



2 - 



1 - 



WASTE 


SUCCESS 


■ 3 


□ 2 


■ 5 


■ 7 


- 


B 4 


- 


□ 8 


■ 1 B 6 




OK 


KILLER 



Figure 5.1 Importance-Performance Grid [Ref. 29] 



In the above example (Figure 4.1), the grid clearly illustrates how the NARDAC 
performed on the various factors and the relative importance of each of those 
factors 1 . Using Alloway’s definition of the quadrants as stated above, the NARDAC 
is performing quite well in the area of System Resources (2) and Staffing (7), which 
are viewed as important to the client’s mission. Therefore, no change in the resource 
allocation is necessary. Accessibility (l)and Responsiveness (6), however, rated 



1 These scores are fictitious and were chosen for illustrative purposes only. 
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very poorly in performance. This rating, on face value, would indicate a need by 
management to focus more attention in those two areas; however, this particular 
client views the two factors as relatively unimportant. Therefore, it would be more 
productive to concentrate on the more important factors, Quality of Products and 
Services (4) and Timeliness of Services (5). These two factors were rated high in 
importance but low in performance and should have management's highest attention. 
According to Alloway, resources should be reallocated in order to bring factors in the 
killer quadrant into the success quadrant. He suggests pulling resources out of the 
waste quadrant. In this area. Cost Management (3) and Timeliness of Services (5) 
rated high in performance but low in importance, therefore efforts to improve 
performance would be wasting scarce resources. 

C. FREQUENCY HISTOGRAM 

Another interesting and informative way to view the data from the questionnaire 
is through the use of frequency histograms (Figure 5.2). Histograms are a very 
good and quick visual tool that can be used to see how all the respondents answered 
a particular item or factor. The frequency histogram fills in the holes where the CSI 
leaves off. To illustrate this point, it is plain to see from Figure 5.2 that the clients 
are quite satisfied with the accessibility of the NARDAC with sixty-five percent of 
the respondents answering satisfied or higher. The histogram could also point out a 
poorly constructed item if there was a very large percentage of not applicable 
responses. 
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FREQUENCY HISTOGRAM FOR ACCESSIBILITY 



N/A = NOT APPLICABLE 

1 = EXTREMELY DISSATISFIED 

2 = DISSATISFIED 

3 = NEUTRAL 

4 = SATISFIED 

5 = EXTREMELY SATISFIED 



N/A 2 3 4 5 

SCORES 



Figure 5.2 Example Frequency Histogram for Accessibility 
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VI. CONCLUSIONS AND RECOMMENDATIONS 



A. CONCLUSIONS 

In conclusion, the research objectives of this study were accomplished. Client 
satisfaction was defined within a domain of eight factors made up of forty-four 
attributes (items). An instrument (questionnaire) was developed to measure these 
factors and an index was constructed to derive an empirical measure of client 
satisfaction. 

Through the process of conducting personal interviews with clients, holding 
discussions with NARDAC management, and digesting numerous articles and 
journals relating to customer satisfaction, client satisfaction was adequately defined 
by eight factors. Forty-four items were developed to further define the factors. A 
self-administered mail questionnaire was developed to collect the data. This method 
of data collection was viewed to be the easiest and least costly, as far as time and 
personnel resources are concerned, to implement by the NARDACs. Due to time 
and resource constraints, the survey instrument was piloted at one site, NARDAC 
San Francisco, using their fifty-two clients as the sample. Twenty-six respondents 
returned the questionnaire, which yielded a return rate of fifty-seven percent after 
accounting for undelivered questionnaires. Next, the pilot data was examined using 
descriptive statistics. As a result of this phase of the questionnaire evaluation, it was 
discovered that two factors exhibited a large number of not applicable responses. To 
alleviate this problem, two items (18 and 28) were revised to broaden their 
applicability. Content validity, or face validity, was achieved by subjecting the 
questionnaire to numerous critiques. The critiques were conducted by the nine 
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NARDAC Client Liaison Officers, faculty members at the Naval Postgraduate 
School, the nine NARDAC Commanding Officers, and by twenty-six client 
organizations during the pilot. Construct validity and reliability tests were also 
performed, but due to the small sample size to item ratio (.59 : 1), the results were 
discounted. Further testing with a larger sample will have to be conducted by the 
NARDACs prior to implementing the survey. Several methods of analyzing the 
survey data were presented in Chapter V. An index was developed to empirically 
represent client satisfaction and a suggested interpretation of the index was provided. 
The index is represented by the following equations: 



Fj = t- Y I ik where 
k=l 

Fj = factor index for factor j 
mj = total number of items for factor j 
L = number of applicable responses 
Li = numeric score for item k of factor j, 

= 1, 2, 3, 4, 5. 

i 8 

CSI = w £ Fj where 
i=l 

CSI = Client Satisfaction Index 
N = number of meaningful factors 

(i.e., factors where at least one item was applicable) 
i = the number of factors. 

An Importance-Performance grid was presented to help NARDAC management 
identify which client satisfaction factors are most important to the clients and how to 
allocate resources accordingly to increase client satisfaction. Another informative 
way to view the data collected from the survey is to use frequency histograms. The 



60 



histograms can easily identify items that are problematic (i.e., an item with a high not 
applicable response rate). As a result of this review, an item may need to be revised 
or maybe even deleted from the questionnaire. 

B. RECOMMENDATIONS 

In this last section of our study, we provide several strongly recommended 
strategies for implementing the measuring instrument that has been developed. In 
addition, the results and conclusions of our study indicate several areas that may be 
pursued for further research. 

I. Implementation Strategies 

The following is a list of recommendations for measuring and analyzing 
client satisfaction for the Navy Regional Data Automation Centers: 

• More rigorous and extensive validation of the measurement instrument should 
be conducted. With a larger sample size (i.e., at least 220 respondents), the 
measurement instrument could be fully tested and evaluated for construct 
validity using the factor analysis method and for reliability using Cronbach's 
alpha. 

• The results of this study and any follow-on actions or plans should be 
published for dissemination to the NARDAC clients. This information could 
help to bolster general acceptance of the instrument by the client community. It 
is especially important that the clients who participated in the interview and the 
pilot receive this feedback to show confidence that their comments were used 
constructively. 

• The results of this study should also be provided to all NARDAC employees 
affected by this study. Such information would help to generate positive 
support and interest in this area and foster general acceptance of the 
measurement instrument. 
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• The measurement instrument should be prepared in booklet form to present a 
more professional appearance. The booklet is bound and not loosely attached 
together. Moreover, this is the norm in industry and professionally prepared 
surveys. 

• Responsibility for the maintenance and administration of the measurement 
instrument and its associated database should be assigned to the Client Liaison 
Officer of each NARDAC. This provides centralized administration of client 
related data and the possible incorporation of the client satisfaction index (CSI) 
into the NARDAC's client profile. 

• The instrument should be initially administered to all NARDAC clients to 
establish a baseline CSI by obtaining 100% participation. From our extensive 
research and lessons learned from the pilot, the questionnaire should be 
delivered to the clients in person by a NARDAC representative (e.g., the Client 
Liaison Officer/staff). This ensures certainty of delivery and provides the 
NARDAC representative the opportunity to update profile information 
regarding the client. The return of the questionnaire may be accomplished by 
mail using pre-addressed envelopes with telephone follow-up, or by collecting 
the questionnaire in person on a predetermined date. To keep the CSI database 
current, the survey should be administered at least biannually and may be 
conducted after periodic Advisory Board or Client Council meetings. 

• A cover letter, such as the one in Appendix C, should be attached in front of 
the questionnaire. 

• The survey results can easily be processed and maintained on a 
microcomputer. A microcomputer-based spreadsheet software package, such 
as LOTUS 1-2-3, can be used to consolidate the data from the questionnaire 
and compute the client satisfaction index. A spreadsheet package with graphics 
capabilities can produce visually enhanced output of the data (e.g., CSI 
histograms, frequency histograms of respondents, or other graphics relating to 
the respondents). 

• The measurement instrument should be used to track client satisfaction over 
time and circumstance. With trend analysis, the effects of technological or 
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policy changes implemented by the NARDACs on client satisfaction could be 
identified and evaluated. 

2. Further Research Areas 

The following is a list of recommendations for further research: 

• The forty-four items of the measurement instrument could be re-examined. 
Further examination of the items could provide insight into the characteristics 
of each item which could further define client satisfaction. This would be 
beneficial in identifying performance elements for the NARDACs. 

• The use of the measurement instrument could be incorporated into a 
productivity model that would include effectiveness and efficiency measures. 
This would allow for the setting of target goals for each NARDAC and the 
measurement of progress toward those goals, [adapted from Ref. 16: p. 191] 

• The measurement instrument could be tailored to other ADP service 
organizations requiring the measurement of client satisfaction. 
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APPENDIX A. MAJOR NARDAC SERVICES 



DATA PROCESSING SERVICES 
TIME SHARING 
DEDICATED 

ANALYSIS AND PROGRAMMING SERVICES 
MICROCOMPUTER 
MINICOMPUTER 
MAINFRAME 

MICROCOMPUTER TRAINING 

ADP SECURITY SERVICES 
TRAINING 

RISK ASSESSMENT ANALYSIS 
DATA COMMUNICATIONS PLANNING SERVICES 
ADP EQUIPMENT PROCUREMENT ASSISTANCE SERVICES 
TECHNICAL CONSULTATION SERVICES 
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APPENDIX B. CLIENT ORGANIZATIONS INTERVIEWED 



SAN DIEGO: 

COMMANDER, NAVAL AIR FORCE, U.S. PACIFIC FLEET 

COMMANDER, NAVAL SURFACE FORCE, U.S. PACIFIC FLEET 

FLEET ACCOUNTING AND DISBURSING CENTER, U.S. PACIFIC FLEET 

NAVAL AIR STATION MIRAMAR 

NAVAL AIR STATION NORTH ISLAND 

NAVAL AVIATION DEPOT NORTH ISLAND 

NAVAL SEA SUPPORT CENTER, U.S. PACIFIC FLEET 

NAVAL STATION SAN DIEGO 

PERSONNEL SUPPORT ACTIVITY SAN DIEGO 

PERSONNEL SUPPORT ACTIVITY NAVAL TRAINING CENTER 

MONTEREY: 

NAVAL POSTGRADUATE SCHOOL 

SAN FRANCISCO: 

COMMANDER, NAVAL BASE SAN FRANCISCO 

MILITARY SEALIFT COMMAND, U.S. PACIFIC FLEET 

NAVAL AIR STATION ALAMEDA 

NAVAL AVIATION DEPOT ALAMEDA 

NAVY SUPPLY CENTER OAKLAND 

PERSONNEL SUPPORT ACTIVITY SAN FRANCISCO 

SHIP INTERMEDIATE MAINTENANCE ACTIVITY ALAMEDA 

WASHINGTON, D.C. : 

NAVAL DATA AUTOMATION COMMAND 
NAVY INSPECTOR GENERAL 
CHIEF OF NAVAL PERSONNEL 
COMPTROLLER OF THE NAVY 

PERSONNEL SUPPORT ACTIVITY, NAVAL DISTRICT WASHINGTON 
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NORFOLK: 



COMMANDER IN CHIEF, U.S. ATLANTIC FLEET 

FLEET ACCOUNTING AND DISBURSING CENTER, U.S. ATLANTIC FLEET 

NAVAL AIR STATION NORFOLK 

NAVAL AVIATION DEPOT NORFOLK 

NAVAL FACILITIES ENGINEERING COMMAND 
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APPENDIX C. CORRESPONDENCE 



17 August 1987 



Commander 

Naval Interview Headquarters 
U.S. Atlantic Fleet 
Norfolk, V A 23150 

Dear Interviewee, 

In a continuing effort to better serve their customers, the Naval Data Automation Command 
(NAVDAC) and the Navy Regional Data Automation Centers (NARDACs) have 
undertaken steps to improve customer satisfaction. As part of this effort, they have 
requested our assistance in developing an index which will be used as a measure of 
customer satisfaction. We are students at the Naval Postgraduate School in the Computer 
Systems Management curriculum. We are conducting this research in partial fulfillment of 
our Master of Science Degree in Information Systems and in hopes of providing NAVDAC 
and the NARDACs with a meaningful and useful tool. 

To develop the customer satisfaction index, we must first identify those critical factors 
which the customers would use to evaluate the services provided by the NARDAC. As 
discussed in our phone conversation, we will be conducting a personal interview with you 
regarding these customer satisfaction attributes on the «date» of August «time». We expect 
the interview to last approximately one hour. Below, is a representative list of factors 
which we feel relate to customer satisfaction. We would appreciate your views on them 
during the interview, or any others that you think may be pertinent. 

• Computer capabilities (mainframe/mini/micro support) 

• Ease of obtaining services 

• Systems design and programming 

• Technical support 

• ADP consultation and assistance 

• Training 

• Cost of ADP services 

• Customer support 

• Timeliness and quality of products and services provided 

If you have any questions concerning the interview or our schedule, please leave a message 
for us at our curriculum office, Autovon 878-2174/2175 or commercial (408) 646- 
2174/2175, and we will contact you as soon as possible. 

We appreciate your assistance in this effort and look forward to meeting you on the <date>. 

Sincerely, 



Prima A. Morris Robert J. Birdwell 

LCDR, USN LT, SC, USN 
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19 October 1987 



Dear Client Liason Officer, 

As pan of the continuing effort to better serve their customers, the NAVDAC Board of 
Directors requested our assistance in developing an index which will be used as a measure 
of customer satisfaction. *We are students at the Naval Postgraduate School in the 
Computer Systems Technology curriculum. The research we are conducting in this area is 
in partial fulfillment of our Masters of Science Degree in Information Systems, and will 
hopefully provide NAVDAC and the NARDACs with a meaningful and useful tool. 

In developing the customer satisfaction index, we have identified those critical factors 
which existing customers would use to evaluate the services provided by the NARDAC. 
We obtained these relevant data through a series of personal interviews from a cross- 
sampling of customers served by NARDAC San Diego, NARDAC San Francisco, 
NARDAC Washington and NARDAC Norfolk. These interviews were conducted in the 
format shown in enclosures (1) and (2). In addition to the interviews, we have also 
conducted comprehensive literature searches, reviewed customer satisfaction 
surveys/questionnaires currently in use by government and commercial organziations, and 
interviewed a number of marketing and client liason representatives in private industry to 
complement our research. 

We value your input and field expeerience in the client liason area. Please review the initial 
draft of the customer satisfaction survey (enclosure (3)) and forward your comments to us 
by 1 November 1987. Enclosed is a pre-addressed, stamped envelope for your 
convenience and the timely delivery of your comments. The return date is critical to our 
research deadline; therefore, we would appreciate your cooperation in helping us meet our 
milestones. 

If you have any questions regarding the survey and/or our research, please leave us a 
message at Autovon 878-2174/2175 or commercial (408)646-2174/2175 and we will return 
your call as soon as possible. 



Sincerely, 

Prima A. Morris, LCDR, USN 
Robert J. Birdwell, LT, SC, USN 
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13 November 1987 



Commanding Officer 
NARDAC 



Dear Captain, 

We are forwarding two (2) drafts (revised) of the NARDAC Client Satisfaction Survey, 
which we developed in response to requirements from the NARDAC Board of Directors, 
for your review and comments. Both questionnaires are identical in content but differ in 
format. As part of our research, please indicate which questionnaire format (long form or 
short form) you prefer and why. As requested by Commander Taylor, please forward 
your comments via electronic mail to NARDAC Pearl Harbor by 23 November 1987. 

If you have questions regarding the questionnaires and/or our research, we may be reached 
at Autovon 878-2174/2175 or commercial (408) 646-2174/2175. 



Very Respectfully, 



Prima A. Morris, LCDR, USN 



Robert J. Birdwell, LT, SC, USN 
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7 December 1987 



Commander 
Naval Client 
U.S. Pacific Fleet 
San Francisco, C A 94130 



Dear Client, 

We are students at the Naval Postgraduate School in the Computer Systems Technology 
Curriculum currently developing a Client Satisfaction Index for the Navy Regional Data 
Automation Centers (NARDACs). The Client Satisfaction Index is designed to measure 
each client's level of satisfaction with the products and services provided by the NARDAC. 
In order to arrive at this measurement, a questionnaire is used as the instrument from which 
the Client Satisfaction Index is derived. This questionnaire was developed based on 
comments from NARDAC clients whom we interviewed in your area and other 
geographical locations. 

Please assist us in validating the Client Satisfaction Survey instrument by completing the 
enclosed questionnaire. We also request your personal critique of the questionnaire. 
Please comment on the following: 

1. Are there any items you didn't understand? 

2. Are there any items you felt were redundant? 

3. Are there any items you feel are critical to client satisfaction that were not 
addressed in the survey? 

4. Was the survey easy to read and complete (format)? 

5. Were the instructions to complete the survey sufficient? 

Please write your comments directly on the survey below the items you are addressing or at 
the end of the survey. We are soliciting your personal comments, therefore a formal reply 
from your command is not required. Your comments and the results of your completed 
survey will be held in strict confidence, to be used only by us to assess the quality and 
validity of the survey instrument. 

In order for us to meet our scheduled milestones, please return your completed survey in 
the enclosed, pre-addressed envelope as soon as possible but not later than 17 December 
1987. If you have any questions regarding the survey or any aspect of our research 
efforts, we may be reached at Autovon 878-2174/2175 or commercial (408)646- 
2174/2175. 

Thank you for your interest and cooperation. 
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Sincerely, 

Prima A. Morris, LCDR, USN 
Robert J. Birdwell, LT, SC, USN 



«NARDAC COMMAND LETTERHEAD» 



Date 



From: Commanding Officer, Navy Regional Data Automation Center, 

To: Commanding Officer, , (Code: XXX) 

Subj: CLIENT SATISFACTION SURVEY 

Enel: (1) Client Satisfaction Survey 

1 . As part of a continuing effort to improve our service, please take a few minutes to 
complete the enclosed questionnaire. 

2. The purpose of the Client Satisfaction Survey is to assist us in determining what 
action can be taken to develop more effective ADP support for our customers. Your 
response will enable us to better understand your present and future needs. The 
questionnaire is designed to snapshot present conditions; therefore, please answer the 
questionaire to reflect current conditions. 

3. We feel this survey will be mutually beneficial. We appreciate your continued 
assistance in helping us serve you. 
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APPENDIX D. INTERVIEW INFORMATION SHEET 



DATE OF INTERVIEW: 

NAME OF ORGANIZATION: 

NAME OF INTERVIEWEE: 

GRADE /RANK: 

CURRENT POSITION: 

LENGTH OF TIME 
INVOLVED WITH NARDAC: 

TYPE OF SERVICE(S): 
NARDAC PROVIDES: 



FREQUENCY OF INTERACTION 
WITH NARDAC (i.e., daily .weekly):. 
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APPENDIX E. INTERVIEW QUESTIONS 



1 . What are the factors or areas you feel that the NARDAC customers should use 
to measure customer satisfaction? 



2. What is, in your opinion, a satisfied NARDAC customer? 



3. Rank the factors in order of importance: 

Computer capabilities (mainframe/mini/micro support) 

Ease of obtaining services 

System design and programming expertise 

Technical support 

ADP consultation and assistance 

Training 

Cost of ADP services 

Customer support 

Timeliness and quality of products and services provided 



4. Specifically, how would you evaluate or measure each of the factors? 
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APPENDIX F. ORIGINAL QUESTIONNAIRE 



1 NA RDAC CUSTOkGR SA TtSFACVON SURVEY 

Pf«83« Place an X in the bo^ which beat describes vour 'evei of satisfa ction with_ each of the loJ lowing statements. 
(Additional comments are welcome). 



.?yggQH 



t . NARDAC's understanding of the impact on your command for late or inaccurate products 
or services. 



Q 0 NOT APPLICABLE 

□ 5 VERY SATISFIED 

□ 4 SATISFIED 

D 3 NEITHER SATISFIED NOR DISSATTSFED 
Cl 2 DISSATISFIED 
O 1 VERY DtSSATISFlED 



2. NARDAC’s familiarity, with vour command’s mission. Cl 0 notappucaoe 

□ 5 VERYSATTSFED 
Q 4 SATISFIED 

□ 3 NEITHER SATTSFED NOR DiSSATTSFEO 

□ 2 OtSSATtSFBD 
D 1 VERY OSSA TISFED 

Cl 0 NOT APPUCASLE 
D 5 VERY SATISFIED 
EH 4 SATISFIED 

Q 3 NEITHER SATTSFED NOR DISSATISFIED 

□ 2 DISSATISFIED 
Q 1 VEHY DISSATTSFED 



3. How satisfied are you with the level of assistance p rovided in preparing proposals for 
new projects? 



4. Flexibility to satisfactonly respond to changes in your specification requirements. Q 0 not applicable 

□ 5 VERY SATT SF C D 

O 4 SATISFIED 

□ 3 NErTHER SATTSFCD NOR OtSSATTSFED 

CH 2 DISSATTSFED 

□ 1 VEHY DISSATTSFED 

5. Courteous and professional attitude of the NARDAC personnel with whom you deal O 0 not a ppljcable 

□ 5 VERYSATTSFED 
EH 4 SATISFIED 

O 3 NEITHER SATTSFED NOR DISSATTSFED 
D 2 DISSATTSFED 
Cl 1 VERY DCSSATTSFED 

C] 0 NOT APPLICABLE 
O 5 VERYSATT SF ED 
Q 4 SATTSFED 

Q 3 NEITHER SATTSFED NOR DtSSATTSFED 
Q 2 DISSATTSFED 
D 1 VERY DISSATTSFED 



6. Promulgation of general purpose information about what is happening around NARDAC 
(i.e., new hardware, system software, application software, training, new personnel, etc.) 
through newsletters, electronic bulletin boards, and Client Counaf Meetings. 
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2 NARDAC CUSTOM R SA TTSPACDON SURVEY 



7. Timeliness of response for a request for services. u 0 NOTAPRJCA8LE 

D 5 VERY SATTSFED 

□ 4 SATtSFH) 

□ 3 NEfTHER SATEFED NOR DtSSATISFED 

D 2 DtSSATISFED 

□ 1 VERY DtSSATISFED 

8. Level of attention given to your project and command. 0 0 not applicable 

□ 5 VERY SATTSFED 

0 4 SATISFIED 

D 3 NEITHER SA T t SF ED NOR DtSSATISFED 

□ 2 DtSSATISFED 

D 1 VERY DtSSATISFED 



9. When a problem associated with a NARDAC product or service ts reported, how satisfied 
are you with the timeliness and Quality of the resolution? 



Q 0 NOT APPLICABLE 

□ 5 VERY SATTSFED 
D 4 SATtSFED 

□ 3 NEfTHER SATtSFED NOR OtSSATlSF ED 

□ 2 DtSSATISFED 

D 1 VERY DtSSATISFED 



Service L evel. Manaaemenl 

10. Accuracy of the initial plannmg/cost estimates for your project request 



1 1 . Ability to adequately explain costs and verify charges . 



1 2. Format and Quality of the Chargeback Report 



D o NOT APPLICABLE 

□ 5 VERY SATTSFED 
0 4 SATtSFED 

□ 3 NEfTHER SATtSFED NOR DtSSATISFED 
D 2 DtSSATISFED 

□ 1 VERY DtSSATISFED 

0 0 NOT APPLICABLE 

□ 5 VERY SATtSFED 

□ 4 SATtSFED 

D 3 NEITHER SATtSFED NOR DtSSATISFED 
D 2 DtSSATISFED 
D 1 VERY DtSSATISFED 

0 0 NOT APPLICABLE 

□ 5 VERY SATt SF ED 

□ 4 SATtSFED 

□ 3 NEfTHER SATtSFED NOR DtSSATISFED 

□ 2 DtSSATTSFED 

□ 1 VERY DtSSATISFED 
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3 hlARDAC CUSTOMER SA TEFACDOH SURVEY 



13. Current rates NAROAC charges for services are competitrve with industry. 



d 0 NOT APPUCABLE 
d 5 HIGK.Y AGREE 
D 4 AGREE 

d 3 NEITT-ER AGREE NOR DtSAGRS 
d 2 DISAGREE 
C3 1 HtGKY OtSAGREE 



14. How satisfied are you with the coordination and follow-uo on tasks/projects assigned to 
subcontractors? 



C] 0 NOT APPUCABLE 
d 5 VERYSATtSFED 
□ 4 SATTSFED 

d 3 NEITHER SATEFED NOR DtSSATEFED 
d 2 DtSSATEFED 
d 1 VERY DtSSATEFED 



d 0 NOT APPUCABLE 
d 5 VERYSATTSFED 
d 4 SATTSFED 

d 3 NERHER SATEFED NOR OtSSATEFED 
d 2 DtSSATEFED 
d 1 VERY DtSSATEFED 

16. Procedures for requesting various NARDAC services. d 0 NOT appucablE 

d 5 VERY SATTSFED 

d 4 SATTSFED 

d 3 NEITHER SATTSFED NOR DtSSATlSFED 

d 2 DtSSATEFED 
d 1 VERY DtSSATEFED 

17. Hours sL operations. d o not appucable 

d 5 VERYSATTSFED 

d 4 SATTSFED 

d 3 NEITHER SATBFED NOR DtSSATlSFED 

d 2 DtSSATEFED 
d 1 VERY DtSSATEFED 

18. Physical location of NARDAC. d 0 NOT APPUCABLE 

d 5 VERY SATEFED 

d 4 SATTSFED 

d 3 NEITHER SATEFED NOR DtSSATlSFED 
d 2 DtSSATEFED 
d 1 VERY DtSSATEFED 



A»«»lhllltv 

15. Communication channels between you and your NAROAC point orf contact. 
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4 HARDAC CUSTOMER £4 TtSFACTION SURVEY 



1 9. Vistor parking spaces at NARDAC. 



□ 0 iscrr APPLICABLE 
Q 5 VERY SATISFED 
0 4 SATISFIED 

□ 3 ItrTHER SATISFED NOP DlSSATTSFED 

□ 2 CXSSATTSFED 

O 1 VERY OtSSATTSFED 



Hardware. Software, and Communication Technology 

20. Current computer hardware capabilities (mainframe, minicomputers, microcomputers). O 0 notappugablE 

D 5 VERY SATISFED 

□ 4 SATtSPED 

□ 3 ICITHER SATISFED NOR OtSSATTSFED 

Q 2 OtSSATTSFED 
O 1 VEHY DSSATISFED 

21 . Ability to meet your promoted automated information system requirements. O 0 not applicable 

O 5 VERY SATISFED 

D 4 SATISFED 

□ 3 NEITHER SATTSFED NOR OtSSATTSFED 

O 2 OtSSATTSFED 
O 1 VERY OtSSATTSFED 



22. Currency and variety of software packages available. 



O 0 NOT APPLICABLE 
O 5 VERY SATISFED 
D 4 SATISFED 

O 3 NEITHER SATISFED NOR DlSSATtSFED 
□ 2 DtSSATTSFED 
O 1 VERY OtSSATTSFED 



23. Telecommunication capabilities and services satisfv/meet my command's operational 
needs. 



O 0 NOT APPLICABLE 
O 5 HGK.Y AGREE 
D 4 AGFS 

O 3 fCTTVER AGREE NOR DISAGREE 
□ 2 DISAGREE 
O 1 HIGHLY DISAGREE 



IfiCimJcal. , PrptlcltQcy, 

24. Level of expertise of the technical staff. 



O 0 NOT applicable 
D 5 VERY SATISFED 
G 4 SATISFED 

O 3 rEm-ER SATISFED NOR DlSSATtSFED 
D 2 DlSSATtSFED 
O 1 VERY OtSSATTSFED 
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5 NARDAC CUSTOMER SA TtSRACTTON SURVEY 



25. The technical staff is well versed in a broad range of ADP subject areas. 



0 0 not applicable 

0 5 HOLY AGREE 

□ 4 AGREE 

C 3 fCfTVER AGREE NOR DISAGREE 

□ 2 DISAGREE 

□ 1 WGH.Y CXSAGREE 



26. The technical staff is keeping abreast of the latest developments in the ADP arena. 



Q 0 NOT applicable 

0 5 HOLY AGREE 
D 4 AGREE 

□ 3 MOTHER AGREE NOR DISAGREE 

□ 2 DISAGREE 

0 1 HOLY OCSAGREE 



27. Ability to design and develop creative and innovative systems. 



□ 0 notappucablE 
D 5 VERYSATISFED 
D 4 SATtSPED 

0 3 NEITHER SATISFED NOR CXSSAT1SFED 
□ 2 DISSATISFIED 
EH 1 VERY DtSSAtTSHED 



28. Personnel assigned to your project demonstrated the proper skill level required to 0 0 notappucablE 

perform the task(s) specified in the Statement of Work. Q 5 vbjysatbfH) 

D 4 SATISFIED 

n 3 NErTHER SATISFED NOR DlSSATtSFED 

0 2 DtSSATt SF ED 

D 1 VERY DtSSATtSFED 

29. Ability of the technical staff to communicate in customer term s/1 anouaoe. Q 0 notappucablE 

D 5 VERYSATISFED 

□ 4 SATTSFED 

Q 3 NEITHER SATTSFED NOR DISSAT1SFED 

□ 2 OSSATtSFED 

□ 1 VERY DtSSATtSFED 



Quality of Predicts and 5ftmcea« 

30. The quality and accuracy of deliverables. 

□ 4 SATISPB) 

O 3 NErTHER SAT1SFED NOR CXSSATtSFED 

□ 2 DfSSATISFK) 

□ 1 VERY DtSSATtSFED 



D 0 NOTAPPUCABLE 
□ 5 VERYSATISFED 
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6 NARDAC CUSTOMER SATISFACTION SURVEY 



31 . Availability of online and batch systems (i.e., the total time the system is up). 



32. Quality of system documentation/manuals (e.g., readable, correct, up-to-date). 



33. The performance of your system(s) currently running on NARDAC computers. 



□ 0 NOTAPPUCABLE 
0 5 VERY SATtSFED 
O 4 SATISFIED 

O 3 NEITHER SATTSFED NOR DtSSATtSFED 
O 2 OtSSATtSFED 
O 1 VERY OtSSATtSFED 

O 0 NOT APPLICABLE 
O 5 VERY SATtSFED 
0 4 SATt SFED 

O 3 NEITHER SATtSFED NOR DtSSATtSFED 
0 2 DtSSATtSFED 
O 1 VERY OtSSATtSFED 

O 0 NOTAPPUCABLE 
O 5 VERYSATtSFED 
O 4 SATtSFED 

O 3 EITHER SATTSFED NOR OtSSATtSFED 
O 2 DtSSATtSFED 
0 1 VERY OtSSATtSFED 



34. NARDAC develops reliable and cost effective application programs for their customers. 



0 0 NOTAPPUCABLE 
0 5 HIGHLY AGREE 
O 4 AGREE 

O 3 NETTVER AGREE NOR DISAGREE 
O 2 DISAGREE 
O 1 HIGH. Y OtSAG REE 



35. Response time of online systems. 0 0 notappucablE 

O 5 VERYSATtSFED 

O 4 SATtSFED 

O 3 NEITHER SATtSFED NOR OtSSATTSFED 

O 2 DtSSATtSFED 
0 1 VERY DtSSATtSFED 

36. Maintenance performed on current systems (Le. t responsiveness in incorporating O 0 notappucablE 

modifications and corrections). 0 5 very satisfied 

0 4 SATTSFED 

0 3 NEITHER SATtSFED NOR DtSSATtSFED 

O 2 DtSSATt SFE D 

0 1 VERY OtSSATtSFED 
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1 7 NARQAC CUSTOM R SATISFACTION SURVEY 

37. Quality of training . 0 0 NOTAPPUCA8LE 

d 5 VERY SATTSFED 

d 4 SATTSFED 

D 3 NEITHER SATTSFED NOR DtSSATISFED 

□ 2 DISSATTSFED 

□ 1 VEHY DISSATTSFED 



□ 0 NOTAPPUCA8LE 
d 5 VEHY SATTSFED 
0 4 SATTSFED 

d 3 NErTVCR SATTSFED NOR DtSSATISFED 
d 2 DISSATTSFED 
d 1 VEHY DISSATTSFED 

39. Adherance to production processing schedules . d 0 NOT APPLICABLE 

d 5 VERY SATTSFED 

d 4 SATTSFED 

d 3 NEITHER SATtSFED NOR DtSSATISFED 

d 2 DISSATTSFED 
d 1 VEHY DtSSATISFED 

40. Timeliness of progress reports . d 0 not applicable 

d 5 VERY SATTSFED 

d 4 SATTSFED 

d 3 NEITHER SATTSFED NOR DtSSATISFED 

d 2 DtSSATISFED 
D 1 VERY DtSSATTSFED 

41 . Timely notification of any delays in output. d 0 not applicable 

d 5 HCH.Y AGREE 

d 4 AGREE 

d 3 NEIT>CR AGREE NOR DtSAGRffi 

d 2 DISAGREE 
d 1 WGH.Y DISAGREE 

42. Reliable and timely delivery of output . d 0 not applicable 

d 5 HCHLY AGREE 

d 4 AGREE 

d 3 NEITHER AGREE NOR DISAGREE 

d 2 DISAGREE 
d 1 HCH.Y DISAGREE 

(Opbonaf): Name Telephone 

PLEASE PflWT 

Thank you for completing this survey. The endosed envelope has been provided for your convenience. 



Timeliness of Products and Services. 

38. Adherance to protect, schedules as specified in the Statement of Work. 
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APPENDIX G. QUESTIONNAIRE SECOND DRAFT 

SHORT FORM 



HARO AC CLIENT SATISFACTION SURVEY 

PLEASE PLACE AN X IN THE SOX WHICH BEST DESCRIBES YOUR LEVEL OF 5 A T7SFA CT1QN WITH EACH OF THE 
FOLLOWING ITEMS, f Additional comments may be made on ike last page). 

USE THE FOLLOWING SCALE: 

N/A NOT APPLICABLE 5 VERY SATISFIED 4 SATISFIED 3 I'CrTHER SATTSFED NOR OtSSATLSFlED 2 DISSATISFIED 1 VERY (DISSATISFIED 



AfcEggsifrllltY. 














1. 


How satisfied are vou with the availability at vour NARDAC ooint of contact. 


n/aO 


n 


40 


30 


n 


«□ 


2. 


The oroceduresfor obtamina various oroducts and service*, 
(refer to NAVDACINST 5230.1) 


N/aO 


sO 


aO 


30 


n 


.□ 


3. 


NAR DAO's current hours of ooeration. 


N/aO 


n 


aO 


30 


n 


o 


4. 


The ohvsrcal location of NARDAC relative to vour command. 


N/aO 


50 


aO 


30 


n 


iO 


5. 


The auantitv of visitor oarkina soaces at NARDAC. 


N/aQ 


sO 


aO 


30 


n 


iO 


System Resources. 

(Encompasses hardware, software, and teleprocessing technology) 














S. 


(i.e., mainframe, minicomputers, microcomputers) 


N/aO 


sO 


aO 


30 


n 


,0 


7. 


NARDAC ability to meet your oroiected automated information system requirements. 


n/aO 


n 


aO 


n 


n 


o 


3. 


The currency and vartatv of software oackaaee available. 


n/aO 


sO 


aO 


n 


n 


,0 


9. 


NARDACs ability to orovide taleorocessma services to meet vour 
command’s operational needs. 


n/aO 


n 


aO 


n 


n 




Cost Management. 














10. 


How satisfied are vou with NARDACs adherence to oroied budaeta? 


n/aO 


n 


aO 


n 


nn. 


11. 


NARDACs ability to adequately MBlain cast and YffrfY CftafOM. 


n/aO 


n 


aO 


n 


n 


lO 


12. 


The format of the Chargeback Report (La., easy to read and understand).. 


n/aO 


n 


aO 


n 


n 


,□ 


13. 


The accuracy of the Chargeback Report 


waO 


£3 


aO 


n 


n 


iO 


14. 


The currant rales NARDAC charaee for products and service*.. 


N/aO 


50 


aO 


n 


n 


iO 
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NAROAC CLIENT SATISFACTION SURVEY 



USE THE FOLLOWING SCALE: 

N/A NOT APPLICABLE 5 VERY SATISFIED 4 SATtSFED 3 NEfT*€R SATISFIED NOR DISSATISFIED 

15. Flexibility in charging for services (e.g., fixed-price contract, tiered rate structure, etc.). 

Qualliv-Pt, Products and.. Services. 

(NARDAC's ability to deliver reliable services and excellent products) 

16. How satisfied are you with the quality of HARD AC software products ? 

1 7. The quality of NARDAC’s data processing services. 

18. The controls over data security . 

1 9. The reliability of online and batch systems (i.e., the total time the system ts up). 

20. The accuracy of system documentaton/manuais. 

21 . The performance of your system(s) currently running on NARDAC computer*. 

22. The response time of NARDAC's online systems. 

23. When a problem associated with a NARDAC product or serves is reorted, how satisfied, 
are you with the Quality fltina.f,8SQlUUPn2 

Ilmiiinm si Strvicfi. 

24. How satisfied are you with NARDAC's adherence to protect schedules as specified in the 
Statement of Work? 

25. Adherence to production processing schedules. 

26. Timeliness of progress fftPOfS- 

27. Software mairtanance p erformed on existing systems (7.#., responsiveness in 
correcting program errors). 

28. Timeliness notificatipn of delays in output 

29. Timeliness of delfYBrafrlaa 

30. Timeliness of response for a request for services. 



2 DISSATISFIED 1 VERY DISSATISFIED 



N/aO 


d 


d 


d 


d 


d 


N/aO 


n 


d 


d 


d 


d 


N/aO 


n 


d 


n 


d 


d 


N/aD 


d 


d 


n 


d 


d 


N/aO 


n 


d 


n 


d 


d 


N/aO 


d 


d 


d 


d 


d 


N/aO 


d 


d 


d 


d 


d 


N/aO 


d 


d 


d 


d 


d 


N/aO 


£3 


d 


d 


d 


d 



N/d 


d 


d 


d 


d 


d 


N/d 


d 


d 


d 


d 


d 


N/d 


d 


d 


d 


d 


d 


N/d 


d 


d 


d 


d 


d 


N/d 


d 


d 


d 


d 


d 


N/d 


d 


d 


d 


d 


d 


N/d 


d 


d 


d 


d 


d 
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NARDAC CLIENT SATISFACTION SURVEY 



USE THE FOLLOWING SCALE: 

N/A NOT APPLICABLE 5 VERY SATISFIED 4 SATtSFED 3 r€fT>€R SATISFED NOR OSSA nSFED 2 DISSAT1SFED 1 VERY OtSSATBFED 



31. 


When a problem associated with a NARDAC product or serves is reported, how satisfied 
are you with the imlmgSS gl.fflfl.rcSQlUlttfl.?- 


n/aD 


£3 


£ 


n 


n 




CgmafttgncY 














32. 


How satislied are vou with the technical staff's level of exoertise?. 

(Technical satff includes computer specialists, programmers, and system analysts) 


n/aD 


£ 


£ 


n 


n 


£ 


33. 


The technical staffs ranae_of_knQwiedae in their resoective ADR fields.. 


N/aO 


£ 


n 


n 


n 


£ 


34. 


The technical staffs knowledae of the latest develooments in their 
respective ADR fields. 


N/aD 


n 


£ 


n 


n 


n 


35. 


NARDAC's ability to desen and develOD creative and innovative systems. 


N/aD 


n 


£ 


n 


n 


£ 


36. 


The NARDAC personnel assigned to your project (i.e..do they demonstrate the 
orooer skill level to oerform the task(s) soecified in the Statement of Work. 


N/aO 


n 


£ 


n 


n 


£ 


Coordination and Communications 














37. 


How satisfied are you with NARDAC*s understanding of the impact on your command for 
late or inaccurate products or services? 


N/aD 


n 


n 


n 


£3 


£ 


38. 


NARDAC's familiarity with vour command's mission 


N/aO 


n 


n 


n 


n 


£ 


39. 


The level of assistance orovided in oreoanna DroDosats for new Droiects 


N/aO 


£ 


£ 


n 


n 


£ 


40. 


Flexibility to satisfactorily reaoond to changes in vour soedficahon reouirementa. 


N/aO 


£3 


£ 


n 


n 


£ 


41. 


Courteous and orofessionaJ attitude of the NARDAC personnel with whom you deal 


N/aO 


£3 


n 


n 


n 


£ 


42. 


Promuioation of aeneral purpose information about whal is haooenina around NARDAC 
|7.e., new Hardware, system software, application software, training, new personnel, etc.) 
through newsletters, electronic bulletin boards, and Client Council meetings. 


N/aO 


£1 


n 


n 


n 


£ 


43. 


The coordination and follow-up on tasks/proiects assigned to subcontractors. 


N/aD 


£ 


£ 


n 


n 


£ 


44. 


(T echmcal satff Modes computer specialists, programmers, and system analysts) 


N/aD 


£3 


n 


n 


n 


£ 
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NARDAC CUENT SATISFACTION SURVEY 



USE THE FOLLOWING SCALE: 

N/A NOT APPLCA8LE 5 VERY SATISRED 4 SATBFED 3 NEITHER SAT1SFCD NOR DtSSATtSFHD 2 DtSSATISFED 1 VERY DISSATISFIED 



PLEASE PLACE AN X IN THE BOX WHICH BEST DESCRIBES THE RELATIVE IMPORTANCE TO YOUR 
ORGANIZATION OF EACH OF THE FOLLOWING PERFORMANCE CRITERIA AS IT RELATES TO CLIENT 
SATISFACTION. 



USE THE FOLLOWING SCALE: 

1. Accessibility 


5 CRITICAL 

sD 


4 IMPORTANT 

u 


3 AVERAGE 

n 


2 NOT iAPORTANT 
£1 


1 IRRELEVANT 
,□ 


2. 


System Resources 


s 0 


a 


n 


n 


iD 


3. 


Cost Msnegement 


n 


n 


n 


n 


i □ 


4. 


Quality of Products end Services 


a 


n 


3 □ 


n 


a 


5. 


Timeliness of Services 


n 


n 


n 


n 


iD 


6. 


Competency 


n 


n 


n 


n 


i □ 


7 


Coordlnetlon and Communications 


n 


n 


n 


n 


iD 



Comminti; 



Nsn e Coda Telephon e 

PLEASE PRNT 

Thank you for completing this survey. The enclosed envelope has been provided for your convenience. 
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APPENDIX H. QUESTIONNAIRE SECOND DRAFT 

LONG FORM 



UAflOAC CL £NT SA T1SFACTK*! SURVEY 

Please place an X in the box which best describes your /eve/ nf mnsfacnnn with each of the following statements; 

(Additional comments art welcome). 

AgCmifrlHtY 

1. How satisfied are von with the availability of vour NARDAC point of contact? D 0 not aprjcaslE 

Q 5 VERY SATTSFH) 

O 4 SATISFIED 

D 3 hEfTVER SATTSFED NOR DISSARSFEO 
n 2 OtSSATTSFEO 
O 1 VERY DtSSATTSFED 

D 0 hCT AFRJCAflLE 
D 5 VERY SATTSFED 
O 4 SATT SFE D 

D 3 ^rTVER SATBFEO NOR DISSATTSFED 
O 2 OBSATTSFED 
D 1 VERY OSSATTSFED 

3. NAROACs current hours of operation Q 0 bCT aprjcaslE 

D 5 VERY SATTSFED 

D 4 SATtSFED 

D 3 NEfTVER SATTSFED NOR DISSATtSFED 

Q 2 DISSATtSFED 
□ 1 VERY DISSATtSFED 



2. The procedures for obtaining various products and services. 
(rotor to NA VDACINST 5230. 1) 



4. The physical location o f NAROAC relauve to your command. 



5. The quantity of visitor parting spaces at NARDAC. 



D 0 NOT APRJCAdLE 
n 5 VERY SATtSFED 
D 4 SATtS FE D 

O 3 fEtTVER SATtSFED NOR DISSATTSFED 
O 2 DSSATtSFED 
Q 1 VERY DSSATTSFED 

D 0 NOT APRJCA8LE 
n 5 VERY SATtSFED 
Q 4 SATtSFED 

O 3 MEITVER SATTSFED NOR OISSATtSFED 
D 2 DISSATtSFED 
□ 1 VERY DISSATTSFED 



Syalem. HgaourcM. 

(Encompasses hardware, software, and teleprocessing technology) 



8 . 



How satisfied are you with NARDACs current 



(l.e.. mainframe, minicomputers microcomputers) ? 



0 0 NOTAPRJCAflLE 
D 5 VERY SATTSFED 
ID 4 SATTSFED 

D 3 tCTTVER SATTSFED NOR OtSSATtSFED 
Q 2 OtSSATTSFEO 
□ 1 VERY DISSATTSFED 
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NAR&AC CIENT SATISFACTION SURVEY 

7. NARDAC's abifrty to meet ycxjr prospered automated information system requirements. Q 0 not a^jcable 

O 5 VERY SATTSFED 

Q 4 SATISFIED 

□ 3 KErTVBR SATTSFED NOR DlSSATtSFED 

Q 2 DtSSATt SF BD 
□ 1 VERY DISSATISFIED 



8. The currency and vanetv of software packages available. 



9. NARDAC's ability to provide teleprocessing services t o meet your command's 

operational needs. 



O 0 NOT APPLICABLE 

□ 5 VERY SATTSFED 
O 4 SATTSFED 

□ 3 tCm-ER SATTSFED NOR OISSATISFED 

□ 2 DtSSATtSFBD 

□ 1 VERY DtSSATtSFBD 

O 0 NOT APPUCA3LE 
D 5 VERY SATtSFBD 

□ 4 SATt SF BD 

□ 3 >dT>CR SATTSFED NOR OtSSATEFED 
O 2 DtSSATtSFBD 

Q 1 VERYDtSSATTSFED 



£aal Management. 

1 0. How satisfied are you with NARDAC's adherence to oroiect budgets ? 



11. NARDAC’s atoilrtv to adeouatelv explain costs and verify charges. 



1 2. The format of the Chargeback Report fie., easy to nmd a/xf understand). 



□ 0 NCTAmCAaE 

□ 5 VERY SATTSFBD 
O 4 SATtSFBD 

□ 3 KBrTVBR SATtSFBD NOR OlSSATTSFED 

□ 2 DCSATt SF BD 

□ 1 VEFYDtSSATISFS) 

□ 0 NOT APRJCA8LE 

□ 5 VERY SATTSFBD 

□ 4 SATTSFBD 

O 3 ►BfTVER SATtSFBD NOR DtSSATTSFBD 
O 2 DBSATTSFED 

□ 1 VEFYDeSATTSFED 



□ 0 NOTAPRXAflLE 

□ 5 VERY SATtSFBD 

□ 4 SATt SF BD 

□ 3 rerTVBR SATTSFBD NOR OlSSATTSFED 

□ 2 DtSSATTSFBD 

□ 1 VERY DISSATTSFED 
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| MARDAC CLEW SAT&ACTKW SUB\GY 

13. The accuracy of the C ha ro aback Report. D 0 not APPucAajE 

0 5 VERY SATTSFED 

0 4 SATISFIED 

0 3 NEfTFER SATTSFED NOR DlSSATlSFED 

D 2 DtSSATTSFED 
D 1 VERY DlSSATlSFED 

14. The current rates NARDAC charges for products and services. Q 0 not appucasle 

D 5 VERY SATTSFED 

0 4 SATTSFED 

D 3 NEITVER SATTSFED NOR DlSSATlSFED 

□ 2 DtSSATTSFED 

Q 1 VERY DtSSATTSFED 

15. Flexibility in charging for services (e.g., fixed -one 9 contract, tiered rate structure, etc.). 0 0 not appucasle 

Q 5 VERYSATTSFE) 

□ 4 SATTSFED 

D 3 JEITFERSATBFED NOR DlSSATlSFED 

□ 2 DtSSATTSFED 

D 1 VERY DtSSATTSFED 



Quality sf Products and, SgfYlctti> 

16. How satisfied are you mth tha qua lily of NARDAC’s software omducts? O 0 not appucahje 

□ 5 VERYSATTSFE) 

n 4 SATTSFED 

□ 3 NEITHER SATTSFED NOR DtSSATTSFED 

□ 2 DtSSATTSFED 

□ 1 VERY DtSSATTSFED 

17. Tha Quality of NARDAC's data processing services. O 0 NOT APPUCAfUE 

D 5 VERYSATBFB) 

D 4 SATTSFED 

‘ D 3 fCTTVER SATTSFED NOR DtSSATTSFED 

□ 2 DtSSATTSFED 

0 1 VERYDeSATBFED 

18. The controls over data security. D 0 not appucasle 

□ 5 VERY SATTSFED 

D 4 SATTSFED 

Q 3 NEITVER SATTSFED NOR DtSSATTSFED 

□ 2 DtSSATTSFED 

□ 1 VERYOBSATBFED 
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NAftDAC CUENT SATISFACTION SURVfTY 



1 9. The reliability o f online and batch systems (i.e., me coray time the system is up). 



20. The accuracy of system document ation/manuab. 



21 . The performance of your system(s) currently running on NARDAC computers. 



22. The response time of online systems. 



□ 0 NOTAmjCAaLE 
O 5 VERY SATtSFED 
D 4 SATtSflED 

D 3 f€fTVeR SATBFHO NOR DlSSATtSFED 
D 2 WSSATSRED 
CD 1 VERY DBSATEFED 

n 0 NOTAPPUCAflLE 
LJ 5 VERY SATISFIED 
CD 4 SA T T Sff ED 

□ 3 f€rrHER SATBFHD NOR DtSSATtSFED 
D 2 DSSATBFCD 

O 1 VERYDSSATtSFH) 

Q 0 NOTAmjCABLE 
O 5 VERYSATTSFE) 

□ 4 SATISFED 

□ 3 fCrtVCR SATBFED NOR CKSSATTSf ED 

□ 2 WSSATtSPED 

□ 1 VBTYWSSATOFB) 

Q 0 HOT APPUCABLE 

□ 5 VERYSATtSFED 

O 4 SATBFED 

□ 3 r€(TVCR SATBFED NOR DtSSATtSFED 

□ 2 DCSSATISRED 

□ 1 VERY DtSSATtSFED 



23. When a problem associated with a NARDAC product or service is reported, how satisfied 
are you wnh the quality of the resolution? 



□ 0 NOT APPLICABLE 

□ 5 VERYSATtSFE) 

O 4 SATT SPE D 

□ 3 fCmCRSATtSFEONOROiSSA-nSPED 

□ 2 DISSATISFIED 

□ 1 VERYOCSATBFB) 



Tlmcilnfiia . of. ■Sandcax 

24. How satisfied are you with NARDACs adherence to project schedules as specified in the 3 0 not applicable 
Statement of Work? Q 5 verysahsfed 

D 4 SATISFIED 

D 3 rCfTTCR SATBFED NOR WSSATtSFED 

O 2 OOSATBPED 

□ 1 VERY DSSATEFB) 



91 



NARDAC CLENT SATISFACTION SURVEY 



25. Adherence to production processing scnedulo*- 



0 0 NOTARRJCABLE 
0 5 VERY SATBFED 
0 4 SATBFED 

O 3 KCmCR SATBFED NOR DlSSATBFED 
O 2 DtSSATBFED 
0 1 VERY WSSATtSFBD 



26. Timeliness of progress reports. 



0 NOT APPLICABLE 
O 5 VERY SATBFED 
0 4 SATBFED 

0 3 NErTVCR SATBFED NOR OBSATBFED 
0 2 OBSATBFED 
O 1 NERYDSSATBFG) 



27. Software Maintenance o artormad on existing systems (7.e., rwsponsrwnts in corracvng 
program amors;. 



0 0 NOT APPLICABLE 
O 5 VERYSATBPED 
O 4 SATBFED 

O 3 NEfTVCR SATBFED NOR DtSSATBFED 
O 2 OBSATBFED 
O 1 VERYOSSATBFED 



28. Timeliness at nopflcapon of delays m output. 



0 0 NOT APPLICABLE 



□ 

□ 

□ 

□ 

□ 



5 VERVSATBFTH) 

4 SATBFED 

3 NEfTVER SATBFED NOR DtSSATBFED 
2 OBSATBFED 
1 VERY DBSATBFB) 



29. Timeliness of deliverables. 0 0 not applicable 

O 5 VERYSATBFS) 

O 4 SATBFED 

0 3 fCTTHER SATBFED NOR OBSATBFED 

0 2 DBSATBFED 

O 1 VERY DBSATBFB) 

30. Timeliness of response for a request for services . 0 0 not applicable 

0 5 VERYSATBFB) 

0 4 SATBFED 

O 3 rcm-ER SATBFED NOR DtSSATBFED 

0 2 OBSA TBFED 
0 1 VERY DBSATBFE) 
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NAROAC ClEbfT SA TISF ACTION SURVEY 



31 . When a problem associated vuth a NAROAC product or service is reporteo, ncrw satisfied 
are you v*th the timeliness of the resolution? 



□ 0 NOTAPPUCAflLE 
O 5 VERY SATBFED 

□ 4 satbhed 

□ 3 fCtTl-ER SATBFED NOR DISSATBFED 
D 2 DBSATBFED 

0 1 VERYD6SATBFED 



S.ampgtgncY 

32. How satisfied are you with the technical staff s level of expertise ? 

(T echntCMi siMff includes computer specialists, programmers, and system analysts; 



! 0 NOTAPFUCAaLE 
□ 5 VERYSATBFB5 
O 4 SATBFED 

O 3 r«n>€R SATBFED NOR DSSATBFED 
O 2 DBSATBFED 
O 1 VERYOBSATBFB) 



33. The technical staffs range of knowledge in their respective ADP field*. 



I 0 not applicable 
O 5 VERY SATBFED 
0 4 SATBFED 

0 3 rCFTVER SATBFED NOR DISSATISFED 

0 2 DCSATBRE D 
0 1 VERY DBSATBFED 



34. The technical staffs knowledge of the latest developments in their respective ADP . 
fields. 



0 0 not applicable 

0 5 VERY SATBFED 
0 4 SATBFED 

0 3 rcn>£R SATBFED NOR DISSATISFED 
0 2 DBSATBFED 
O 1 VERY DBSATBFED 



35. NARDAC’s ability to design and develop creative and innovative s ystems. 0 0 notaprjcablE 

0 5 VERY SATBFED 

O 4 SATBFED 

0 3 EITTER SATBFED NOR DISSATISFED 

O 2 DBSATBFED 
O 1 VERY DBSATBFED 

36. The NAROAC personnel assigned to your project, (i.e., do they demonstrate the 0 0 notaprjcablE 

proper skill level t o perform the task(s) specified in the Statement of Work. 0 5 v^ty satwh) 

O 4 SATBFED 

O 3 rcmER SATBFED NOR OBSATBFED 

O 2 DBSATBFED 

O 1 VERY DBSATBFED 
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NARDAC CLENT SATISFACTION SURVEY 



gflflidimigfl and Communlcatlgm 

37. How satisfied ar» you with NARDAC’s understanding of the impact on your command for 

laie or inaccurais amflucis or amcaa? 



D 0 NOT APPUCABLE 

D 5 VERY SATBFED 
O 4 SATBFED 

O 3 *EfTVER SATBFED NOR DtSSATBFED 
O 2 DSSATBFED 
□ 1 VERY DtSSATBFED 



38. nardac-s familiamy with ) 



I 0 NOT APPLICABLE 
Q 5 VERY SATBFED 
D 4 SATBFED 

□ 3 SERVER SATBFED NOR DtSSATBFED 
D 2 DSSATBFED 

□ 1 VERYOOSATBFB5 



39. The lave! of assistance p rovided In preparing proposals for new profeesa. 



0 notappucable 

□ 5 VERYSATTSFBD 
Q 4 SATBFED 

D 3 *ETTVER SATBFED NOR DtSSATBFED 
O 2 OSSATBFED 

□ 1 VERY DBSATBFED 



40. Flexibiirry to satisfactorily respond to changes in your specification requirements. 



□ 0 notappucable 

D 5 VERY SATBFED 

□ 4 SATBFED 

D 3 MERVER SATBFED NOR DtSSATBFED 

□ 2 DSSATBFED 

D 1 VERY DSSATBFED 



41. Courteous and professional attitude of trie NARDAC Personnel with whom you deal 



Q 0 notappucable 

D 5 VERY SATBFED 
□ 4 SATBFED 

O 3 NEfT>ER SATBFED NOR DtSSATBFED 
0 2 DSSATBFED 
O 1 VERY DSSATBFO 



42. Promulgation of general purpose information about wftat Is happening around NARDAC 
(i.e.. now hardware, system software, appUcatron software, training, new personnel ere.; 
through newsletters, electronic bulletin boards, and Client Council Meetings. 



□ 0 notappucable 

C 5 VERY SATBFED 

□ 4 SATBFED 

D 3 ICmCR SATBFED NOR DtSSATBFED 

□ 2 DtSSATBFED 

O 1 VERY DSSATBFED 
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NARDAC CUENT SATISFACTION SURVEY 

43. The coordination and follow-op on tasKs/oroiscts assigned to subcontractors. 



□ 0 MJT APPUCAflLE 
d 5 VERY SATBFED 
d 4 SATBFED 

□ 3 KCITHER SATBFED NOR DiSSATlSFED 
d 2 DtSSATTSFE) 
d 1 VERY OtSSATJSFED 



44. The amity of nardac's tachracai staff to communicate in ctiftrn terma'lancuaci. 

(T echmcaS stiff includes computer specialists, programmers, and system analysts) 



LJ 0 not applicable 
□ 5 VERY SATBFED 
Q 4 SATBFED 

Q 3 KERVCR SATBFED NOR DBSATBFED 
d 2 DBSATBFED 
Q 1 VERY DBSATBFED 



Please place an X in t fie box which best describes the relative importance of each of the following performance criteria: 



1. Accessibility 



O 5 critical 

O 4 MPORTANT 
□ 3 AVERAGE 
O 2 NOT IMPORTANT 
Q 1 ^RELEVANT 



2. System Resource 



Q 5 critical 

Q 4 IMPORTANT 

□ 3 AVERAGE 

O 2 NOT IMPORTANT 

□ 1 ivclevant 



3. Cost Management 



d 5 CRfnCAL 
Q 4 IMPORTANT 
□ 3 AVERAGE 
O 2 NOT IMPORTANT 
d 1 FtfELEVANT 



4. Quality of Products 
end Services 



Q 5 CRITICAL 
Q 4 WPORTANT 
□ 3 AVERAGE 
Q 2 NOT IMPORTANT 
d 1 ^RELEVANT 



5. 



Timeliness of 
Service 



d 5 CRITICAL 
d 4 IMPORTANT 
d 3 AVERAGE 
d 2 NOT IMPORTANT 
d 1 ^RELEVANT 



6. Competency 



d 5 CRITICAL 
d 4 IMPORTANT 
d 3 AVERAGE 
d 2 NOT IMPORTANT 

dl 



RELEVANT 



7. Coordination end 
Communicetione 



d 5 CRITICAL 
d 4 IMPORTANT 
d 3 AVERAGE 
d 2 NOT IMPORTANT 
d 1 ^RELEVANT 



Nhm » Qari a Tebchon e 

REASEPWfT 

Thank you for completing this survey. The enclosed envelope has been provided for your convenience . 
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APPENDIX I. PILOT POPULATION 



Naval Surface Force, U.S. Pacific Fleet, San Diego 

Fleet Accounting and Disbursing Center U.S. Pacific Fleet, San Diego 

Fleet Hospital Support Office, Alameda 

Naval Station,Treasure Island, San Francisco 

Naval Air Station, Lemoore 

Naval Air Systems Command, Naval Air System Command HQ, Washington, D.C. 

Naval Biosciences Laboratory, Oakland 

Naval Strike Warfare Center, Naval Air Station, Fallon 

Shipbuilding,Conversion and Repair USN, Hunters Point, San Franciscco 

Naval Weapons Station, Concord, CA 

Naval Supply Systems Command, Washington, D.C. 

SIMA NRMF San Francisco, NAS Alameda 

USS Enterprise CVN-65 

Naval Base, San Francisco 

Navy Public Works Center San Francisco Bay 

Mare Island Naval Shipyard, Vallejo 

Naval Education and Training Financial Information Processing Center, NAS 
Pensacola 

Navy Accounting and Finance Center, Washington, D.C 

Naval Air Station, Alameda 

Naval Reserve Force, New Orleans 

Naval Air Station, Moffett Field 

Service Group 1, Oakland 

Shipbuilding, Conversion and Repair USN, Seattle 
U.S. Maritime Defense Zone Pacific, Alameda 
Navy Military Personnel Command, Washington, D.C. 

Naval Aviation Depot, Naval Air Station, Alameda 
Navy Comptroller Standard System Activity, Pensacola 
Naval Supply Center, Oakland 
Personnel Support Activity, San Francisco 
Military Sealift Command Pacific, Oakland 
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Naval School Physical Distribution Management, Oakland 

Naval Medical Command, Northwest Region, Oakland 

Special Boat Unit Eleven, San Francisco 

Naval Postgraduate School, Monterey 

Naval Facilities Engineering Command, Alexandria, VA 

USS Kiska 

Naval Supply Center Puget Sound, Bremerton, WA 

Joint Military Postal Activity Pacific, San Francisco 

Marine Barracks, Naval Air Station, Alameda 

Sacramento Army Depot, Sacramento 

Navy Astronautics Group, Point Mugu, CA 

Chief Preservation Assistance Branch, San Francisco 

Navy Resale and Services Support Office, Field Support Office, Oakland 

Navy Legal Service Office, San Francisco 

Mare Island Naval Shipyard, Vallejo 

Naval Station Mare Island, Vallejo 

Tactical Electronic Warfare Squadron One Twenty Nine, NAS Whidbey Island 
Defense Subsistence Region Pacific, Alameda 
Naval Air Station, Fallon 

Naval and Marine Corps Reserve Center, San Francisco 
Naval Electronic Systems Engineering Center Vallejo 
NARDAC Newport 
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APPENDIX J. PILOT QUESTIONNAIRE 



! NAROAC CUENT SATlSf action survey 

PLEASE PLACE AN X IN THE BOX WHICH BEST DESCRIBES YOUR LEVEL OF_ SATISFACTION WITH EACH OF THE 
FOLLOWING ITEMS. ( Additional comments may be made on the last page). 

use Twe following scale: 

N/A NOT APPUCA8LE 5 EXTREKCLY SADSFCO 4 SATISFIED 3 NtUTRAL 2 DISSATISFIED 1 ©CTREVCLY DBSAT1SF60 

Accessibility* 

(Ease of obtaining same*) 



1. 


Tha availabilltv of vour NAROAC noint of contact 


huD 


n 


40 


n 


n 


,□ 


2. 


Tha ora cad ur as reauired inNA VDACINST 5230. 1 for obtajnma various NAROAC oroducts 
and sarvtcas. 


N/aO 


n 


40 


£3 


n 


o 


3. 


The currant hQUfS Of QgflrallQn at NAROAC. 


N/aO 


n 


40 


30 


n 


o 


4. 


Tha Dhv3icaj location of NAROAC ralativa to vour command. 


n/aO 


sO 


o 


n 


n 


o 


5. 


TTia quantity of YisitacoartmasQacga at naroac. 


N/aO 


sO 


40 


n 


n 


.□ 



S xaiam B&afluifiu, 



(Encompasses narbwars, software, and t et epro cess ing technology) 



6. 


Tha currant commit ar hardwara capabilities at NAROAC. 
(i.e., mainframe, minicomputers, microcomputers) 


n/aO 


n 


40 


n 


20 


10 


7. 


Tha ability of NAROAC to maat your PTBlflgttd automated information 3Y3tam ttOUmiflflQ. 


n/aO 


n 


40 


n 


n 


iO 


3. 


Tha currency and van a tv of software packages availabia at NAROAC. 


n/aO 


n 


40 


30 


n 


iO 


9. 


Tha ability art NAROAC to orovida adaauata talaorocassina servtcas to maat vour 
command's opa rational needs. 


n/aO 


n 


40 


30 


n 


0 


Co-U- Management. 

(Includes controls over project budgets, rates charged for services bdSng procedures, 
and price structures ) 














10. 


NARDAC’3 ability to provide an accurate assessment of both 
tha costs and tha schaduia for a protect 


*n 


n 


40 


30 


n 


,0 


11. 


NAROACs adherence to oroiect budgets. 


waD 


n 


40 


30 


20 


10 


12. 




N/aO 


n 


40 


30 


n 


iO 


13. 


Tha format of NAROACs Chargeback Raport (Is, easy to reed and understand X 


N/aD 


n 


40 


30 


20 


iO 


14. 


Tha accuracy of NAROACs Charg aback Raport 


N/aO 


n 


40 


30 


20 


iO 
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NARDAC CLIENT SATISFACTION SURVEY 



USE THE FOLLOWING SCALE: 



N/A NOT APPLICABLE 5 EXTREMELY SATTSFED 4 SATTSHED 3 NEUTRAL 2 DISSATISFIED 1 EXTREMELY DSSAT1SFED 



1 S. The current rates NARDAC charges tor product* and services. 


n/aO 


£3 


£3 


n 


n 


10 


1 6. Flexibility in charama tor NARDAC oroducts and service* 
(e.g., fixed-pnc e contract, tiered rate structure, ate.). 


ntaO 


£3 


£3 


3 0 


£ I 


i0 


Quality Qf-Products and Services. 

(NARDAC's ability to deliver reliable services and excellent products) 














17. The Quality ot NARDAC developed application system products. 


n/aO 


£3 


n 


30 


20 


0 


18. The quality ot NARDAC’s daUargCgSSfflg SgfYICflS. 


n/aQ 


n 


£3 


£ I 


20 


0 


1 9 . The security controls over vour data that is maintained on NARDAC comDuters. 
(i.e. t physical security, user td/passwords. etc.) 


N/aO 


n 


£3 


£1 


20 


0 


20. The re ii ability of NARDAC online and batch systems |7.e.. the total time the system a up). 


N/aO 


n 


£1 


£3 


20 


0 


21 . The accuracy of NARDAC’s system documentafcon/manuais. 


N/aO 


£3 


£3 


30 


20 


10 




N/aO 


£1 


£3 


30 


20 


i0 


23. The Quality of the resolution to a orobJem associated with a NARDAC orodixs or service 


N/aO 


£3 


£3 


30 


20 


lO 


Timeliness of Services. 

(NARDAC's ability to be punctual and * schedule conscientious ' with is products and services) 














24. NARDAC’s adherence to oroiect schedules aa soedfied in the 
Project Request. 


N/aO 


£3 


£1 


30 


n 


ID 


25. The ability of NARDAC to adhere to Droduetion orocessina schedules. 


n/aD 


n 


n 


30 


n 


0 


26. The freauenev of NARDAC oroaress reoorts. 


N/aO 


n 


£3 


£1 


20 


iD 


27. The timely notification o< delays in output 


N/aO 


£2 


£3 


n 


n 


lO 


28. Timeliness of deliverables from NARDAC. 


N/aG 


n 


n 


n 


n 


iD 


B.gj>pan,glv.gnggi 

(How quickly NARDAC reacts or responds to a eat needs ) 














correcting program errors). 


N/aD 


n 


£3 


30 


n 


O 
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HARD AC CLIENT SATISFACTION SURVEY 



USE THE FOLLOWING SCALE: 



UlA NOT APPLICABLE 5 EXTR£L€LY SATTSRED 4 SAT1SHED 3 NEUTRAL 2 DISSATISFCD 1 EXTREMELY DISSATISFIED 



30. 




n/aO 


n 


40 


£3 


n 


iD 


31. 


The reponsiveness of NARDAC to resolving a reported problem associated with a product 
or service. 


n/aD 


n 


40 


£1 


n 


£ 


32. 


The flexibility of NARDAC to changes in your project requirements. 


n/aD 


n 


40 


n 


n 


iD 


33. 


The level of assistance orovided in oreoanno oroMsals for new Droiects. 


n/aG 


£3 


40 


n 


n 


£ 


Staffing. 

(Encompasses the resourcefulness and requisite qualifications of NARDAC personnel) 














34. 


The overall ranae of excertiso within the NARDAC oroan cation. 


n/aD 


n 


40 


n 


n 


i0 


35. 




n/aO 


n 


40 


n 


n 


i0 


36. 


The abilitv of NARDAC Derconnel to keeo abreast of the latest develooments in their 
respective functional area. 


N/aD 


n 


40 


n 


n 


£ 


37. 


The courteous and orofessional attitude of the NARDAC oersonnel with whom vou deaL 


N/aD 


n 


40 


n 


n 


iD 


38. 


The technical staffs abilitv to deson and deveioo creative and innovative systems. 
(T echmcal staff includes computer specialists, programmers, and system analysts) 


N/aD 


n 


40 


n 


n 


,0 


39. 


The technical staffs ability to satisfactorily analyse and document vour oroiect 
requirements. 


N/aO 


n 


40 


n 


n 


iO 


40. 


The ability of nardacs technical staff to ammunicaig in ciit nt tarms/lanauagt, 


N/aO 


n 


40 


n 


n 


iD 


General Business Practices. 














( The manner in which NARDAC conducts business with ks clients) 














41. 


NARDACs familiarity with vour command's mission as it reiaies to data orocessmo. 


N/aO 


n 


40 


n 


n 


lO 


42. 


nardac* undemanding of ffia impact on your command for laia or inaccurate products 
or services. 


N/aO 


n 


40 


n 


n 


lO 


43. 




N/aO 


n 


40 


n 


n 


iO 


44. 


(i.e., new hardware, system software, application software, training, new personnel, etc) 
through newsletters, electronic bulletin boards, and Client Counal meetings. 


N/aD 


n 


40 


n 


n 


lO 
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4 NAROAC CLIENT SATISFACTION SURVEY 



PLEASE PLACE AN X IN THE BOX WHICH BEST DESCRIBES THE RELATIVE IMPORTANCE TO YOUR 
ORGANIZATION OF EACH OF THE FOLLOWING PERFORMANCE CRITERIA AS IT RELATES TO CLIENT 
SATISFACTION. 



: THE FOLLOWING SCALE: 


5CRrriCAL 


4 IMPORTANT 


3 AVERAGE 


2 NOT IMPORTANT 


1 IRRELEVANT 


Accessibility 

(Ease of obtaining service) 


n 


4 □ 


n 


n 


43 


System Resources 
(Encompasses hardware, software, and 
teleprocessing technology) 


5 □ 


n 


a 


£3 


n 


Cost Management 

(Includes controls over project budgets, rates 
changed for services, billing procedures, and price 
structures ) 


5 □ 


n 


n 


n 


n 


Quality of Products end Services 
INARDAC's ability to deliver reliable services and 
excellent products) 


n 


n 


n 


n 


n 


Tlmsllnsss of Services 

(NARDAC's ability to be punctual and * schedule 

conscientious 0 with its products and services) 


n 


a 


n 


n 


d 


Responsiveness 

(How quickly NARDAC reacts or responds to 
dent needs) 


sO 


n 


n 


n 


43 


Staffing 

( Encompasses the resourcefulness and requisite 
qualifications of NARDAC personnel) 


5 □ 


n 


n 


n 


n 


Gsnsrsl Business Practices 
( The manner in which NARDAC conducts business 
with its clients) 


n 


43 


n 


n 


n 



Comminti; 



Nam e Oris Telephon e 

RJEASEPRNT 

Name of yotr o roa nc ato n N mti# of vour NARDAC PO C 

Thank you for completing this survey. The enclosed envelope has been provided for your convenience. 
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APPENDIX K. REVISED QUESTIONNAIRE 



HARO AC CLIENT SATISFACTION SURVEY 

PLEASE PLACE AN X IN THE BOX WHICH BEST DESCRIBES YOUR LEVEL OF SATISFACTION WITH EACH OF THE 
FOLLOWING ITEMS. (Additional comments may be made on the last page). 

U3C THE FOLLOWING SCALE: 

WA NOT APPLICABLE 5 EXTREMELY SATTSFED 4 SATISPED 3 NEUTRAL 2 OtSSARSFED 1 EXTREK®. Y 0CSSAT1SFHD 



Accessibility. 

(Ease of obtaining service) 

1 . Tha availability of your NARDAC point of contact 

2. The BfQCrtUfgS IflflUliad InNAYDAClNST 522QJ. lor obtaining various NARDAC products 
and services. 

3. The currant hours of operation at NARDAC. 

4. The ohvsicaJ location o f NARDAC relative to your command. 

5. Tha quantity cf visitor parking spaces at naroac. 

Sxaigni. Resources. 

i Encompasses hardware, software, and tetepnocesseig technology ) 

6. Tha currant CCmPUlSf hardware capabilities at NARDAC. 

(i.e.. mainframe, mmcomputers. microcomputers ) 

7. Tha ability of NARDAC to maat your QTOieflfllaulPinalBti information SYStSfll jafllii£gffiSOtt> 

3. Tha currency and vanetv of softwara package* available at NARDAC. 

9. Tha ability of NARDAC to provida adequate taloorpcassino sarvicas to maat your 
command's opa rational needs. 

Cost Management. 

(Includes controls over project budgets, rotes charged for services, billing procedures, 
and price structures ) 

1 0. NARDACs ability to provida an aocurata assassmant at both 
tha costs and tha schaduia for a projacL 

1 1 . NARDACs adherence to QTPiflCt budget! 

1 2. Tha ability of NAROAC to adequately explain oroiact casts. 

13. Tha forma! of NAROACs Chargeback Raport (Le. easy to reed and understand). 

1 4. Tha accuracy of NARDACs Chargeback Report. 



N/aO 


n 


u 


n 


n 


,□ 


WaO 


n 


n 


n 


n 


a 


n/aO 


a 


u 


n 


a 


,□ 


N/aO 


n 


u 


n 


a 


U 


N/aO 


n 


n 


n 


n 


,□ 


n/aO 


n 


n 


n 


n 


n 


N/aO 


n 


a 


n 


n 


,□ 


N/aO 


n 


a 


n 


n 


iO 


n/aO 


n 


u 


n 


n 


a 



waO 


n 


a 


n 


n 


a 


N/aO 


sO 


u 


n 


n 


,□ 


N/aO 


n 


n 


n 


n 


iD 


n/aO 


n 


n 


n 


n 


a 


N/aO 


n 


a 


n 


n 


,□ 
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NAROAC CUEMT SATISFACTION SURVtY 



USE THE FOLLOWING SCALE: 

N/A NOT APPLICABLE 5 EXTHETyELY SATEFED 4 SATKFED 3 NEUTRAL 2 OtSSATTSFED 1 EXTTELEIY DISSATTSFED 



1 5. The current rates NAROAC charges tor products and service*. 



1 6. Flexibility in charging for NARDAC products and service*. 
(e.g., fixed-pnce contract, tiered rare structure, etc.). 



Quality— gi-Prsdugltt anti StrYicti. 

(NARDAC's ability to deliver reliable services and excellent products) 

17. The duality of NARDAC developed application system products. 



18. The duality of NARDAC's services 

(Services indude training. Inf or mat ion Resource Center (IRC) assistance, 
processing, Client Liaison services. studies. etc.) 

1 9 . The security controls over vour data that is maintained on NARDAC computer*. 

(i.e., physical security , user td/passwords, etc.) 

20. The reliability o f NARDAC online and batch system* (i.e., the total time the system is up). 



21 . The accuracy of NARDAC’s system document ation/manuais. 

22. The response time of NARDAC’s online systems. 

23. The duality of the resolution to a problem associated with a NARDAC product or service. 
Timeliness of Services. 

(NARDAC's ability to be punctual and ’ schedule conscientious ' with its products and services) 

24. NARDAC’s adherence to ormect schedules as sped fied in the 
Project Request. 

25. The ability of NARDAC to adhere to production OroCttSinfl fiShftdulM, 

26. The frequency of NARDAC PfPflfSailtflgQA. 



27. The timely notification of delays in outauL 



28. Timeliness of services from NARDAC. 

(Services include training. Inform at ion Resource Center (IRC) assistance, 
processing, Client Liaison services.studies.etc.) 
fliafignaiYtnftM 

(How quickly NARDA C reacts or responds to dent needs) 

29. Software maintenance p erformed on existing NARDAC systems (Le., responsiveness in 
correcting program errors;. 

30. The responsiveness of NARDAC to s request for services. 



N/aD 


n 


n 


n 


£2 


iD 


N/aD 


n 


n 


n 


£2 


,□ 


n/aO 


n 


n 


n 


£2 


n 


n/aD 


sD 


n 


£2 


£2 


iD 


n/aO 


£3 


u 


n 


£2 


iD 


n/aO 


n 


a 


£2 


2 □ 


iD 


n/aD 


n 


a 


£2 


£2 


iD 


n/aD 


n 


o 


£2 


£2 


iD 


n/aO 


n 


n 


n 


n 


n 


n/aO 


n 


n 


£2 


£2 


iD 


N/aO 


n 


n 


£2 


n 


,□ 


N/aO 


sO 


n 


£2 


£2 


iD 


n/aO 


sO 


n 


£2 


£2 


a 


n/aO 


n 


n 


£2 


£2 


,□ 


N/aO 


sG 


n 


30 


n 


iD 


N/aO 


n 


n 


£2 


£2 


iD 
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HARD AC CLIENT SATISFACTION SURVEY 



USE THE FOLLOWING SCALE.* 

N/A NOT APPLICABLE 5 EXTPELCLY SATBFED 4 SAT1SFED 3 NEUTRAL 2 OtSSATISFED 1 EXTREMELY DtSSATTSFED 



31. 


The reponsiveness of NARDAC to resolving a reported problem associated with a product 
or service. 


N/aD 


50 


40 


n 


n 


n 


32. 


The flexibility of NARDAC to changes in your project requirements. 


n/aD 


sO 


n 


30 


n 


,□ 


33. 


The level of assistance orovided in oreoannq orooosals for new oroiects. 


n/aD 


sO 


n 


30 


n 


i0 


Staffing. 

(Encompasses the resourcefulness and requisite qualifications of HARD AC personnel) 














34. 


The overall ranoe of exoertisa within the NARDAC oraamzahon. 


n/aD 


n 


n 


30 


n 


i0 


35. 


The level of exDerttse of the NARDAC oeraonnel with whom vou deaL 


N/aO 


n 


40 


30 


n 


i0 


36. 


The ability of NARDAC Dersonnel to keep abreast of the latest devebDments in their 
respective functional area. 


N/aD 


n 


40 


30 


n 


i0 


37. 




N/aO 


n 


40 


30 


n 


i0 


38. 


The technical staff's ability to desian and deveioo creative and innovative systems. 
(T echmcai staff includes computer specialists, programmers, and system analysts) 


N/aO 


n 


40 


30 


n 


i0 


39. 


The technical staff's ability to satisfactorily analyze and document vour oroiect 

rMuirgments. 


N/aD 


n 


40 


n 


n 


U 


40. 


The ability of nardacs technical staff to cpmmunicaifl in diant larms/languaga- 


N/aO 


n 


40 


n 


n 


iO 


General Business Practices. 














(The manner in which NARDAC conducts business with its diems) 














41. 




N/aQ 


n 


40 


n 


n 


lO 


42. 


or services. 


n/aD 


sO 


40 


n 


n 


iO 


43. 




N/aD 


n 


40 


n 


n 


lO 


44. 


(i.e.. new hardware, system software, application software, training, new personnel, etc) 
through newsletters, electronic bulletin boards, and Client Council meetings. 


N/aO 


n 


40 


n 


n 


lO 
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HARD AC CLIENT SATISFACTION SURVEY 



PLEASE PLACE AN X IN THE BOX WHICH BEST DESCRIBES THE RELATIVE IMPORTANCE TO YOUR ORGANIZATION 
OF EACH OF THE FOLLOWING PERFORMANCE CRITERIA AS IT RELATES TO CUENT SATISFACTION. 



• Twe following scale: 


5 CRITICAL 


4 ILRORTANT 


3 average 


2 NOT IMPORTANT 


1 IRRELEVANT 


Accessibility 

(Ease of obtaining service) 


£3 


£3 


£3 


£3 


i □ 


System Resources 
( Encompasses hardware, software , and 
teleprocessing technology) 


£3 


£3 


£3 


£3 


,□ 


Cost Mensgement 

(Includes controls over project budgets, rates 
charged for services, billing procedures, and price 
structures) 


n 


£3 


£3 


£3 


n 


Quality of Products end Services 
[NARDAC's ability to oeliver reliable services and 
excellent products) 


£ 3 


£3 


£3 


£3 


n 


Timeliness of Services 

(NARDACs ability to be punctual and ' schedule 

conscientious' with its products and services) 


£3 


£3 


£3 


£3 


a 


Responsiveness 

(How quickly NARDAC reacts or responds to 
ctent needs) 


£3 


£3 


£3 


£3 


n 


Stsfflng 

(Encompasses the resourcefulness and requisite 
qualifications of NARDAC personnel) 


£3 


£3 


£3 


£3 


o 


General Bueiness Prectlces 

(The manner m whch NARDAC conducts busmen 

with its clients) 


£3 


£3 


£3 


£3 


iD 



Comminti; 



tome Ccd» Teaphone 

R£ASEPHNT 

tamed volt o ma n gaia r tarns d your NARQAC PO C 

Thank you for completing this survey. The enclosed envelope has been provided for your convenience. 
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